He\ dZddlmZddlZddlZddlZddlZddlZddlZddl Z ddl m Z ddl m Z ddlmZmZe e j"ZddZGdd ej(Zdd ZGd d Zej0Zdd Zy)z"Better tokenizing for coverage.py.) annotationsN)Iterable)env)TLineNoTSourceTokenLinesc #Kd}d}d}|D]\}}\}}\}} } ||k7r|r|jdrd} |jdrd} n4|tjk(r!d|vr|jdd d ddk(rd} | rBt |jdd d z } t j d d|| f|| d zf|| }|t jt jfvr|}t j ||||f|| f| |}yw)aBReturn all physical tokens, even line continuations. tokenize.generate_tokens() doesn't return a token for the backslash that continues lines. This wrapper provides those tokens so that we can re-create a faithful representation of the original source. Returns the same values as generate_tokens() Nz\ T\F ri) endswithtokenSTRINGsplitlentokenize TokenInfoNEWLINENL) toks last_line last_lineno last_ttextttypettextslinenoscolelinenoecolltextinject_backslashccols 5/usr/lib/python3/dist-packages/coverage/phystokens.py _phys_tokensr's<!IKJAE'=uow ' !Y//7 $( &&t,',$ell*u}T1)=a)@)D)L,1(#yt4R89A=D",,v $'46):! I ))8;;7 7J  $QVWW O'sDD cfeZdZdZddZej dk\rd dZej dk\rd dZyy) SoftKeywordFinderzCHelper for finding lines with soft keywords, like match/case lines.cjt|_|jtj|yN)setsoft_key_linesvisitastparse)selfsources r&__init__zSoftKeywordFinder.__init__Rs!,/E 399V$% c|jj|j|jD]1}|jj|jj3|j |yz Invoked by ast.NodeVisitor.visitN)r-addlinenocasespattern generic_visit)r1nodecases r& visit_MatchzSoftKeywordFinder.visit_MatchXsY    # #DKK 0  =##'' (;(;< =   t $r4)r6 cp|jj|j|j|yr9)r-r:r;r>)r1r?s r&visit_TypeAliasz!SoftKeywordFinder.visit_TypeAlias`s(    # #DKK 0   t $r4N)r2strreturnNone)r?z ast.MatchrFrG)r?z ast.TypeAliasrFrG) __name__ __module__ __qualname____doc__r3sys version_inforArDr4r&r)r)Ps9M&  7" % 7" %#r4r)c#:Ktjtjtjtj h}g}d}|j djdd}t|}tjjrt|j}t|D]d\}}\}} \} } } d} tj d|D]3} | dk(r |g}d}d}n| dk(rd}n||vrd}n | r | |kDr|j#d d | |z zfd} tj$j'|d j)d d }|tj*k(rt-j.|rd}nyt0j2dk\rftjjrLt-j4|r7t7|dk(rd}nt7|dk(r|ddd k(rd}nd}|r|vrd}|j#|| fd}d} 6sc| }g|r|y y w)aGenerate a series of lines, one for each line in `source`. Each line is a list of pairs, each pair is a token:: [('key', 'def'), ('ws', ' '), ('nam', 'hello'), ('op', '('), ... ] Each pair has a token class, and the token text. If you concatenate all the token texts, and then join them with newlines, you should have your original `source` back, with two differences: trailing white space is not preserved, and a final line with no newline is indistinguishable from a final line with a newline. rz r Tz( )Fr ws xxNr6keyr5r )rINDENTDEDENTrrr expandtabsreplacegenerate_tokensr PYBEHAVIOR soft_keywordsr)r-r'rerappendtok_namegetlowerNAMEkeyword iskeywordrLrM issoftkeywordr)r2 ws_tokenslinecoltokgenr-rrsliner _r" mark_startpartmark_end tok_classis_start_of_lines r&source_token_linesrpfs u||U]]HKKHI"$D C   q ! ) )&$ 7F V $F ~~##*62AA5A&5I'1umudYa HHVU+# Dt|   )# $*KKsdSj'9 :;!&J$--11%>DDFrJ EJJ&((/$) ))W4>>77G> !!DN{{4(11H #'(@(@(J#K t !%s #A22 A<N)rFrG)r{rErF TokenInfos)rHrIrJrKr3rYrNr4r&rrrrs 8  r4rrczt|jdj}tj|dS)zDetermine the encoding for `source`, according to PEP 263. `source` is a byte string: the text of the program. Returns a string, the name of the encoding. Tr)iter splitlines__next__rdetect_encoding)r2rys r&source_encodingrs5F%%d+,55H  # #H -a 00r4)rr|rFr|)r2rErFr)r2bytesrFrE)rK __future__rr/rwrbr\rLrrtypingrcoveragercoverage.typesrrrr|r' NodeVisitorr)rprrrYrrNr4r&rsy)"  5h(( ) 4n%%,DN  8"#33 1r4