o {#iXã@sdZddlZddlZddlmZddlmZmZmZm Z m Z m Z ddl m Z mZddlmZmZddlmZmZe e¡ZGdd „d ƒZd ed efd d „Zd ed e efdd„Zd eded e efdd„Zd eded efdd„Zd ed efdd„Zd ed efdd„Zd ed e efdd„Z d efdd„Z!de de efdd „Z"de d!e#fd"d#„Z$de d!e#fd$d%„Z%de&fd&d'„Z'd(e&d)efd*d+„Z(d(e&d)e#fd,d-„Z)d(e&d)e#fd.d/„Z*d(e&d)efd0d1„Z+d(e&d)e#fd2d3„Z,d(e&d)e#fd4d5„Z-d(e&d)e#fd6d7„Z.d>de&d8e#fd9d:„Z/d;ed efdÚ signatureÚjs_funcÚnameÚargumentr(r(r)Ú get_signatureLs ÷ zCipher.get_signaturerCcCsVt d¡|jD]}t |¡}| |¡}|r$| ¡\}}|t|ƒfSqtddd‚)aèParse the Javascript transform function. Break a JavaScript transform function down into a two element ``tuple`` containing the function name and some integer-based argument. :param str js_func: The JavaScript version of the transform function. :rtype: tuple :returns: two element tuple containing the function name and an argument. **Example**: parse_function('DE.AJ(a,15)') ('AJ', 15) zparsing transform functionrArr) r5r6rrrrÚgroupsr3r )r$rCrÚregexÚ parse_matchÚfn_nameÚfn_argr(r(r)rAjs     þÿzCipher.parse_functionN) Ú__name__Ú __module__Ú __qualname__Ústrrr@r=rFr rr3rAr(r(r(r)rs  rrr?cCsZgd¢}t d¡|D]}t |¡}| |¡}|r&t d|¡| d¡Sq tddd‚)zÖExtract the name of the function responsible for computing the signature. :param str js: The contents of the base.js asset file. :rtype: str :returns: Function name from regex match ) zX\b[cs]\s*&&\s*[adf]\.set\([^,]+\s*,\s*encodeURIComponent\s*\(\s*(?P[a-zA-Z0-9$]+)\(zg\b[a-zA-Z0-9]+\s*&&\s*[a-zA-Z0-9]+\.set\([^,]+\s*,\s*encodeURIComponent\s*\(\s*(?P[a-zA-Z0-9$]+)\(zi(?:\b|[^a-zA-Z0-9$])(?P[a-zA-Z0-9$]{2})\s*=\s*function\(\s*a\s*\)\s*{\s*a\s*=\s*a\.split\(\s*""\s*\)zS(?P[a-zA-Z0-9$]+)\s*=\s*function\(\s*a\s*\)\s*{\s*a\s*=\s*a\.split\(\s*""\s*\)z1(["\'])signature\1\s*,\s*(?P[a-zA-Z0-9$]+)\(z!\.sig\|\|(?P[a-zA-Z0-9$]+)\(z€yt\.akamaized\.net/\)\s*\|\|\s*.*?\s*[cs]\s*&&\s*[adf]\.set\([^,]+\s*,\s*(?:encodeURIComponent\s*\()?\s*(?P[a-zA-Z0-9$]+)\(z>\b[cs]\s*&&\s*[adf]\.set\([^,]+\s*,\s*(?P[a-zA-Z0-9$]+)\(zM\b[a-zA-Z0-9]+\s*&&\s*[a-zA-Z0-9]+\.set\([^,]+\s*,\s*(?P[a-zA-Z0-9$]+)\(zH\bc\s*&&\s*a\.set\([^,]+\s*,\s*\([^)]*\)\s*\(\s*(?P[a-zA-Z0-9$]+)\(úS\bc\s*&&\s*[a-zA-Z0-9]+\.set\([^,]+\s*,\s*\([^)]*\)\s*\(\s*(?P[a-zA-Z0-9$]+)\(rPzfinding initial function nameú"finished regex search, matched: %sr-Úget_initial_function_nameÚmultipler)r5r6rrrrr )rÚfunction_patternsrrHÚfunction_matchr(r(r)rRŠs    þÿrRcCs4t t|ƒ¡}d|}t d¡t||dd d¡S)a‹Extract the "transform plan". The "transform plan" is the functions that the ciphered signature is cycled through to obtain the actual signature. :param str js: The contents of the base.js asset file. **Example**: ['DE.AJ(a,15)', 'DE.VR(a,3)', 'DE.AJ(a,51)', 'DE.VR(a,3)', 'DE.kT(a,51)', 'DE.kT(a,8)', 'DE.VR(a,3)', 'DE.kT(a,21)'] z.%s=function\(\w\){[a-z=\.\(\"\)]*;(.*);(?:.+)}zgetting transform planr-)rú;)rÚescaperRr5r6r Úsplit)rrDrr(r(r)r®s rr'cCsZdt |¡}t d¡tj|tjd}| |¡}|s!td|d‚| d¡  dd¡  d ¡S) aåExtract the "transform object". The "transform object" contains the function definitions referenced in the "transform plan". The ``var`` argument is the obfuscated variable name which contains these functions, for example, given the function call ``DE.AJ(a,15)`` returned by the transform plan, "DE" would be the var. :param str js: The contents of the base.js asset file. :param str var: The obfuscated variable name that stores an object with all functions that descrambles the signature. **Example**: >>> get_transform_object(js, 'DE') ['AJ:function(a){a.reverse()}', 'VR:function(a,b){a.splice(0,b)}', 'kT:function(a,b){var c=a[0];a[0]=a[b%a.length];a[b]=c}'] zvar %s={(.*?)};zgetting transform object)ÚflagsÚget_transform_objectrr-r,ú z, ) rrWr5r6rÚDOTALLrr rÚreplacerX)rr'rrHÚtransform_matchr(r(r)rZÈs   rZcCs<t||ƒ}i}|D]}| dd¡\}}t|ƒ}|||<q |S)aWBuild a transform function lookup. Build a lookup table of obfuscated JavaScript function names to the Python equivalents. :param str js: The contents of the base.js asset file. :param str var: The obfuscated variable name that stores an object with all functions that descrambles the signature. ú:r-)rZrXÚ map_functions)rr'Útransform_objectÚmapperÚobjrDÚfunctionÚfnr(r(r)rès  rc CsØdg}t d¡|D][}t |¡}| |¡}|ret d|¡t| ¡ƒdkr-| d¡S| d¡}|re| d¡}t dj t  | d¡¡d|¡}|re| d¡ d¡  d ¡}d d „|Dƒ}|t |ƒSq t d d d‚)zúExtract the name of the function that computes the throttling parameter. :param str js: The contents of the base.js asset file. :rtype: str :returns: The name of the function used to compute the throttling parameter. zha\.[a-zA-Z]\s*&&\s*\([a-z]\s*=\s*a\.get\("n"\)\)\s*&&\s*\([a-z]\s*=\s*([a-zA-Z0-9$]+)(\[\d+\])?\([a-z]\)z Finding throttling function namerQr-r.z[]zvar {nfunc}\s*=\s*(\[.+?\]);)Únfuncú,cSsg|]}| ¡‘qSr()Ústrip)Ú.0Úxr(r(r)Ú %sz0get_throttling_function_name..Úget_throttling_function_namerSr)r5r6rrrr2rGrrhÚformatrWrXr3r )rrTrrHrUÚidxÚarrayr(r(r)rlÿs6ø      ÿý€ÿrlcCsZt t|ƒ¡}d|}t |¡}| |¡}t|| ¡dƒ d¡}d |¡}|  d¡|S)zãExtract the raw code for the throttling function. :param str js: The contents of the base.js asset file. :rtype: str :returns: The name of the function used to compute the throttling parameter. z%s=function\(\w\)r-r,r0r) rrWrlrrr ÚspanrXr7r)rrDÚ pattern_startrHÚmatchÚcode_lines_listÚ joined_linesr(r(r)Úget_throttling_function_code-s    ruc CsXt|ƒ}d}t |¡}| |¡}t|| ¡ddƒ}t|ƒ}g}|D]s}z | t|ƒ¡Wq#t y7Ynw|dkrB| d¡q#|  d¡rV|  d¡rV| |dd…¡q#|  d¡r‘dt fd t fd tfd tfd tfd tfdtfdtfdt ff } d} | D]\} } t | |¡r| | ¡d} q|| r‘q#| |¡q#tt|ƒƒD] } || dur©||| <q|S)z«Extract the "c" array. :param str js: The contents of the base.js asset file. :returns: The array of various integers, arrays, and functions. z,c=\[r-ÚnullNú"rrdzS{for\(\w=\(\w%\w\.length\+\w\.length\)%\w\.length;\w--;\)\w\.unshift\(\w.pop\(\)\)}ú{\w\.reverse\(\)}z{\w\.push\(\w\)}z.;var\s\w=\w\[0\];\w\[0\]=\w\[\w\];\w\[\w\]=\w}z case\s\d+z1\w\.splice\(0,1,\w\.splice\(\w,1,\w\[0\]\)\[0\]\)z;\w\.splice\(\w,1\)}zL\w\.splice\(-\w\)\.reverse\(\)\.forEach\(function\(\w\){\w\.unshift\(\w\)}\)zBfor\(var \w=\w\.length;\w;\)\w\.push\(\w\.splice\(--\w,1\)\[0\]\)}FT)rurrrr rprÚappendr3Ú ValueErrorÚ startswithÚendswithÚthrottling_unshiftÚthrottling_reverseÚthrottling_pushÚthrottling_swapÚthrottling_cipher_functionÚthrottling_nested_spliceÚ js_spliceÚthrottling_prependr1r2)rÚraw_codeÚ array_startÚ array_regexrrÚ array_rawÚ str_arrayÚconverted_arrayÚelrbÚfoundrrer8r(r(r)r!FsZ   þ  ÷   €  €r!c Cs t|ƒ}d}t |¡}| |¡}t|| ¡ddƒ}d}t |¡}| |¡}g} |D]"}|ddkrB|  |d|d|df¡q+|  |d|df¡q+| S)a·Extract the "throttling plan". The "throttling plan" is a list of tuples used for calling functions in the c array. The first element of the tuple is the index of the function to call, and any remaining elements of the tuple are arguments to pass to that function. :param str js: The contents of the base.js asset file. :returns: The full function code for computing the throttlign parameter. ztry{r-z(c\[(\d+)\]\(c\[(\d+)\](,c(\[(\d+)\]))?\)ér0r)rurrrr rpÚfindallry) rr…Útransform_startÚ plan_regexrrÚtransform_plan_rawÚ step_startÚ step_regexÚmatchesÚtransform_stepsr(r(r)rˆs      rÚarrÚ_cCs|ddd…S)a@Reverse elements in a list. This function is equivalent to: .. code-block:: javascript function(a, b) { a.reverse() } This method takes an unused ``b`` variable as their transform functions universally sent two arguments. **Example**: >>> reverse([1, 2, 3, 4]) [4, 3, 2, 1] Nrr()r–r—r(r(r)Úreverse«sr˜r+cCs ||d…S)zÓAdd/remove items to/from a list. This function is equivalent to: .. code-block:: javascript function(a, b) { a.splice(0, b) } **Example**: >>> splice([1, 2, 3, 4], 2) [1, 2] Nr()r–r+r(r(r)Úsplice¿s r™cCs>|t|ƒ}tt||g|d|…|dg||dd…ƒƒS)zùSwap positions at b modulus the list length. This function is equivalent to: .. code-block:: javascript function(a, b) { var c=a[0];a[0]=a[b%a.length];a[b]=c } **Example**: >>> swap([1, 2, 3, 4], 2) [3, 2, 1, 4] r-rN)r2r@r)r–r+Úrr(r(r)ÚswapÐs 2r›cCs4| ¡ddd…}tt|ƒƒD]}||||<qdS)zÈReverses the input list. Needs to do an in-place reversal so that the passed list gets changed. To accomplish this, we create a reversed copy, and then change each indvidual element. Nr)Úcopyr1r2)r–Ú reverse_copyr8r(r(r)r~âsÿr~ÚdÚecCs| |¡dS)zPushes an element onto a list.N)ry©ržrŸr(r(r)rîsrcCs|t|ƒt|ƒt|ƒS)zÞPerform the modular function from the throttling array functions. In the javascript, the modular operation is as follows: e = (e % d.length + d.length) % d.length We simply translate this to python here. )r2r r(r(r)Úthrottling_mod_funcósr¡cCsFt||ƒ}|| d…|d| …}| ¡|D]}| |¡qdS)z©Rotates the elements of the list to the right. In the javascript, the operation is as follows: for(e=(e%d.length+d.length)%d.length;e--;)d.unshift(d.pop()) N)r¡Úclearry)ržrŸÚnew_arrr‹r(r(r)r}þs  ÿr}c Cs€tdƒ}d}t|ƒ}| ¡}t|ƒD]+\}}| |¡| ||¡|d|t|ƒ}| ||¡||||<|d8}qdS)a“This ciphers d with e to generate a new list. In the javascript, the operation is as follows: var h = [A-Za-z0-9-_], f = 96; // simplified from switch-case loop d.forEach( function(l,m,n){ this.push( n[m]=h[ (h.indexOf(l)-h.indexOf(this[m])+m-32+f--)%h.length ] ) }, e.split("") ) z@ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_é`é r-N)r@rœÚ enumerateÚindexr2ry) ržrŸÚhÚfÚthisÚ copied_listÚmÚlÚ bracket_valr(r(r)r s,ÿ  úrcCs2t||ƒ}t||d|dƒ}t|dd|dƒdS)aèNested splice function in throttling js. In the javascript, the operation is as follows: function(d,e){ e=(e%d.length+d.length)%d.length; d.splice( 0, 1, d.splice( e, 1, d[0] )[0] ) } While testing, all this seemed to do is swap element 0 and e, but the actual process is preserved in case there was an edge case that was not considered. r-rN)r¡rƒ)ržrŸÚ inner_splicer(r(r)r‚-s üür‚cCsbt|ƒ}t||ƒ}|| d…|d| …}| ¡|D]}| |¡qt|ƒ}||ks/J‚dS)a; In the javascript, the operation is as follows: function(d,e){ e=(e%d.length+d.length)%d.length; d.splice(-e).reverse().forEach( function(f){ d.unshift(f) } ) } Effectively, this moves the last e elements of d to the beginning. N)r2r¡r¢ry)ržrŸÚ start_lenr£r‹Úend_lenr(r(r)r„Qs  r„cCs*t||ƒ}|d}|||d<|||<dS)z6Swap positions of the 0'th and e'th elements in-place.rN)r¡)ržrŸr©r(r(r)r€ps   r€ÚstartcGs¸z|t|ƒkr t|ƒ}|dkrt|ƒ|}Wn ty!d}Ynw|r,|t|ƒ|kr2t|ƒ|}||||…}|d|…t|ƒ|||d…}| ¡|D]}| |¡qR|S)a¬Implementation of javascript's splice function. :param list arr: Array to splice :param int start: Index at which to start changing the array :param int delete_count: Number of elements to delete from the array :param *items: Items to add to the array Reference: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/splice # noqa:E501 rN)r2Ú TypeErrorr@r¢ry)r–r²Ú delete_countÚitemsÚdeleted_elementsr£r‹r(r(r)rƒxs"  € þ $ rƒrCcCsJdtfdtfdtfdtff}|D]\}}t ||¡r|Sqtddd‚)z For a given JavaScript transform function, return the Python equivalent. :param str js_func: The JavaScript version of the transform function. rxz{\w\.splice\(0,\w\)}z9{var\s\w=\w\[0\];\w\[0\]=\w\[\w\%\w.length\];\w\[\w\]=\w}zD{var\s\w=\w\[0\];\w\[0\]=\w\[\w\%\w.length\];\w\[\w\%\w.length\]=\w}r`rSr)r˜r™r›rrr )rCrbrrer(r(r)r`¢sþø  ÿ r`)N)1Ú__doc__ÚloggingrÚ itertoolsrÚtypingrrrrrrÚpytube.exceptionsr r Úpytube.helpersr r Ú pytube.parserr rÚ getLoggerrLr5rrOrRrrZrrlrur!rr˜r3r™r›r@r~rr¡r}rr‚r„r€rƒr`r(r(r(r)Ús>   o$ .B#   "$*