?dCndZddlmZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z ddl mZdd l mZdd l mZdd l mZd+d Zd ZdZdZdZdZdZdZdZdZdZdZGddeZej@dddgZ!ej@dddgZ"d Z#d!Z$d"Z%ga&d#Z'd$Z(d%Z)d&Z*d'Z+d(Z,d)Z-d*Z.y),aDecide what the format for the code should be. The `logical_line.LogicalLine`s are now ready to be formatted. LogicalLInes that can be merged together are. The best formatting is returned as a string. Reformat(): the main function exported by this module. )unicode_literalsN)pytree)token) pytree_utils)format_decision_state) format_token) line_joiner)style)verifierclg}d}tjd}t|D]v}|j}t ||j ||||j z}t j||} | j|js|jjr4|jjj|j_ nI|jjr3|jjj|j_ |r-|jr!t|j|j|td|j Dr t#||js t%|r$t'|t)|||t+| nt-|s t/|rt)|||t+| nt1|rItd|j Ds-| j2ru| j5dd| j2r nTt7| sIt j||} | jt'|t)||dt+| |j9||}yt;|t=||S)ahReformat the logical lines. Arguments: llines: (list of logical_line.LogicalLine) Lines we want to format. verify: (bool) True if reformatted code should be verified for syntax. lines: (set of int) The lines which can be modified or None if there is no line range restriction. Returns: A string representing the reformatted code. N INDENT_WIDTHc34K|]}|jywN is_comment.0toks :/usr/lib/python3/dist-packages/yapf/yapflib/reformatter.py zReformat..Is 4S^^ 4c34K|]}|jywr)must_break_beforers rrzReformat..Ws 2K69362G2G2KrFnewlinedry_run)r Get_SingleOrMergedLinesfirst_FormatFirstTokendepthrFormatDecisionStateMoveStateToNextTokendisablervaluerstriplast+_RetainRequiredVerticalSpacingBetweenTokensanytokens$_RetainVerticalSpacingBeforeComments_LineHasContinuationMarkers_RetainHorizontalSpacing_RetainRequiredVerticalSpacing_EmitLineUnformatted%_LineContainsPylintDisableLineTooLong_LineContainsI18n_CanPlaceOnSingleLine next_tokenAddTokenToState_AnalyzeSolutionSpaceappend_AlignTrailingComments_FormatFinalLines) llinesverifylines final_lines prev_line indent_widthlline first_token indent_amtstates rReformatrC's!+)>*,#F+0e++Kk5;; ;G +J ! 5 5eZ HE  ==   !KK--446  ::  ::++224  y(( 4EKK49 ; 4u|| 4 4,U3 }}3E:u%$UIu=5! / 6 E "$UIu=5! u %c2K=B\\2K/K    eU;   #5 )$77zJe   "u%$UIt<5!uIa0d% ; //c|jD]2}|j|jj|j4y)z-Retain all horizontal spacing between tokens.N)r*RetainHorizontalSpacingrcolumnr!)liners rr-r-ms3 [[?c 1 14::>?rDcd}| |j}|jr t}|jD]}t ||||}y)z*Retain all vertical spacing between lines.N)r'r$setr*r()cur_liner=r;prev_tokcur_toks rr.r.ssL (~~H  EEg/5IHrDch|y|jr)|j|jjdz}nR|jr:|j j s|j j}n|j}n |j}|jr)|j|jjdz }n |j}|js9|jjdr||jjdz }||z }|jr |jsns|rq|jt||dzrS|jjd}t|dz|}t|j|}t||z |}|j|y)zDRetain vertical spacing between two tokens if not in editable range.N \) is_stringlinenor%count is_pseudoprevious_tokenis_multiline_stringrendswith intersectionrangewhitespace_prefixlenmaxAdjustNewlinesBefore) rMrLr; prev_lineno cur_linenorequired_newlinesdesired_newlineswhitespace_linesdeletable_liness rr(r(sm   //HNN$8$8$>>K  " " 6 6++22kOOk//K '--"5"5d";;JJ   !8!8!>8>>''--K ;.  3 3 ##E+zA~$FG0066t<[1_j9%,,-=>?O-?,. 01rDcd}|jD]_}|jrO|rM|j|jj dz |jz dkDr|j t |}ay)z(Retain vertical spacing before comments.NrOrQ)r*rrSr%rTr^ONE_BLANK_LINE)rH prev_tokenrs rr+r+s_* [[c ~~* ciiood+ +j.?.? ?! C   0J rDc^|jr|jj}|j}|js |jr||j j dz }|jrd}n|jj|kD}|j|d|jryy)aEmit the line without formatting. The line contains code that if reformatted would break a non-syntactic convention. E.g., i18n comments and function calls are tightly bound by convention. Instead, we calculate when / if a newline should occur and honor that. But otherwise the code emitted will be the same as the original code. Arguments: state: (format_decision_state.FormatDecisionState) The format decision state. rOFrN) r3rVrSrWrRr%rTis_continuationr4)rBrVprevious_linenors rr/r/s %%44N$++O))^-E-E--33D99o%%g  ''/9g '59 rDctjdrS|jD]D}|jst j tjd|j sDytjdrxt|j}t|dz D]R}|j|dzj dk(s#|j|j tjdvsRyy)aoReturn true if there are i18n comments or function calls in the line. I18n comments and pseudo-function calls are closely related. They cannot be moved apart without breaking i18n. Arguments: line: (logical_line.LogicalLine) The line currently being formatted. Returns: True if the line contains i18n comments or function calls. False otherwise. I18N_COMMENTTI18N_FUNCTION_CALLrQ(F) r rr*rrematchr%r\rZ)rHrlengthindexs rr1r1s YY~{{ BHHUYY~%> J  YY#$  Fvz" ++eai & &# - ++e  " "eii0D&E E rDcVtjd|jjS)zBReturn true if there is a "pylint: disable=line-too-long" comment.z#\bpylint:\s+disable=line-too-long\b)rosearchr'r%rHs rr0r0s 9499?? KKrDc:td|jDS)z7Return true if the line has continuation markers in it.c34K|]}|jywr)rirs rrz._LineHasContinuationMarkers..s 8SS 8r)r)r*rus rr,r,s 8DKK 8 88rDc|jDcgc]}|j}}tjdrd|vrytjd|jz}|j }d}|j s|js |jr|j}d}|y|j|ztjd kxr td |jd|D Scc}w) zDetermine if the logical line can go on a single line. Arguments: line: (logical_line.LogicalLine) The line currently being formatted. Returns: True if the line can or should be added to a single line. False otherwise. FORCE_MULTILINE_DICTLBRACEFr NT COLUMN_LIMITc34K|]}|jywrrrs rrz(_CanPlaceOnSingleLine.. sES#..Er) r*namer rr!r'is_pylint_commentis_pytype_commentis_copybara_commentrV total_lengthr))rHx token_namesrAr' last_indexs rr2r2s"&-A-+- ii&'H ,C yy(4::5* $*  6 6    DJ \    j (EIIn,E E FEDKK ,DEE EG.sC*c d}|t|krB||}|jsJd}|jD]}|jst|jt s,|j jdsHg}d} |t|zt|k(rn.||t|z}|jsJ|r)|jdjjdrn|jr|jgd}g} |jD]} | j} | jd} | dk7rt|t|}d}| | d zd } | jr| jt|k|d j| | j z }| rt|t| }|j| J|d z }d } |jD] }||kDs |} n| |} t|D]+\}} | s |||z}d}|jD]} | js|t| ksJ| || ksJd | | |z d z z}|d z }g}t| j jdD]B\}}|jd j||j!|dk(s;d | d z z}Ddj#|}| jj%d}|j|r|t|d }|| _|t| k(r,J|t|z }d}n|s|d z }|t|krAy y )z+Align trailing comments to the same column.rF#Tz rOr{rQNz{}{} )r\r*r isinstancespaces_required_beforelistr% startswithformatted_whitespace_prefixr$r6rfindr]format enumeratesplitstripjoinlstrip)r<final_lines_indexrHprocessed_contentrall_pc_line_lengthsmax_line_length this_line line_contentpc_line_lengthsline_tokr[ newline_index aligned_col potential_colall_pc_line_lengths_indexpc_line_length_index whitespacecomment_line_index comment_lineexisting_whitespace_prefixs rr7r7 sC ,, ( )D ;;;{{r ..Z(B(BDI ))  s #! %8!9 9S=M M !"3c:M6N"NO)!! !!!q!==HHP      & &r * ,/#** Oh ( D D -33D9M" #OS5F Gol"3MA4E4F"G""$$S%67fmm,=x~~NNl O!/33GHO  $ $_ 5SX 1  77 M _ ,'K    '+;D ;!) > 6 % !"36O"OP)!" #** ,h"")C,@@@@$%9:[HHH0D EEIKj"a'"l6?..&&t,7.72$l##FMM*2>2D2D2F%HI&*"kAo6* 7"YY|4l 44;;DA)(()CD+C0J,K,LM +hn? ,B&_)== ==S) >V S!455  erh 1w C ,,rDcg}|D]$}g}|jD]}|js7|j|j|j|jF|j j jdrl|j j jdr|jjdk(s|j jdvs|jd|jdj||s tj|d'dj|dzS)z2Compose the final output from the finalized lines.rOr:z,}])rr{) r*rUr6rr%r3r[rrVrr VerifyCode)r<r:formatted_coderHformatted_liners rr8r8s..dN{{% ]]c==>cii(00;;DA00;;C@    $ $ + NN  .    $%"''.12 .,-.  4 ''rDceZdZdZdZdZy) _StateNodea]An edge in the solution space from 'previous.state' to 'state'. Attributes: state: (format_decision_state.FormatDecisionState) The format decision state for this node. newline: If True, then on the edge from 'previous.state' to 'state' a newline is inserted. previous: (_StateNode) The previous state node in the graph. cJ|j|_||_||_yr)ClonerBrprevious)selfrBrrs r__init__z_StateNode.__init__sDJDLDMrDcNdj|j|jS)Nz%StateNode(state=[ {0} ], newline={1}))rrBr)rs r__repr__z_StateNode.__repr__s" 4 ; ; DLL ""rDN)__name__ __module__ __qualname____doc__rrrDrrrs "rDrOrderedPenaltypenaltyrT QueueItemordered_penalty state_nodec rd}t}g}t|dd}tj|t t d|||dz }|r|d}|j j}|j}|jjsntj||dkDrd|j_ t|}|j|j|t|k(rt||d||}t||d||}|r|syt!|tj|jy)aAnalyze the entire solution space starting from initial_state. This implements a variant of Dijkstra's algorithm on the graph that spans the solution space (LineStates are the nodes). The algorithm tries to find the shortest path (the one with the lowest penalty) from 'initial_state' to the state where all tokens are placed. Arguments: initial_state: (format_decision_state.FormatDecisionState) The initial state to start the search from. Returns: True if a formatting solution was found. False otherwise. rFNrQi'T)rJrheapqheappush _QueueItem_OrderedPenaltyrrrrBr3heappopignore_stack_for_comparisonr\add_AddNextStateToQueue_ReconstructPath) initial_staterTseenp_queuenodeitemrbefore_seen_counts rr5r5s % $ ' M5$ /$..*_Q%>EF1*% 1:D""**G ??D ::   MM' u}/3djj,D HHTZZCI% !$ug FE $eW EE- 0  =%--"8"C"CD rDc N|jj}|r|jj|s|S|s|r|St|j||}||jj |d|z }t j |tt||||dzS)aYAdd the following state to the analysis queue. Assume the current state is 'previous_node' and has been reached with a penalty of 'penalty'. Insert a line break if 'newline' is True. Arguments: penalty: (int) The penalty associated with the path up to this point. previous_node: (_StateNode) The last _StateNode inserted into the priority queue. newline: (bool) Add a newline if True. count: (int) The number of elements in the queue. p_queue: (heapq) The priority queue representing the solution space. Returns: The updated number of elements in the queue. T)rr must_splitrQ) rB MustSplitCanSplitrr4rrrr)r previous_noderrTrrrs rrrs""",,.* ]((11*= L Z L M''- @$ TZZ ' 't (<<'..*_We%DdKL rDctj}|jr*|j||j}|jr*|D]}|j |j d!y)a6Reconstruct the path through the queue with lowest penalty. Arguments: initial_state: (format_decision_state.FormatDecisionState) The initial state to start the search from. current: (_StateNode) The node in the decision graph that is the end point of the path with the least penalty. FrN) collectionsdequer appendleftr4r)rcurrentpathrs rrrsg    $OOGG Gd!!$,,!FGrDc *tr3td|kDr'tjtr td|kDr'd}t|r-ts|gan#td|krd}tj||j t ||||||y)aFormat the first token in the logical line. Add a newline and the required indent before the first token of the logical line. Arguments: first_token: (format_token.FormatToken) The first token in the logical line. indent_depth: (int) The line's indentation depth. prev_line: (list of logical_line.LogicalLine) The logical line previous to this line. final_lines: (list of logical_line.LogicalLine) The logical lines that have already been processed. r{FT) indent_levelN) NESTED_DEPTHpop _IsClassOrDefr6AddWhitespacePrefix_CalculateNumberOfNewlines)r@ indent_depthr=r< first_nesteds rr r 4s b)L8 b)L8,; "^l b L (l,' !! lI!,l<"!rDrQrc|jdvry|jxr*|jdk(xr|jjdk(S)N>defclass@Tasyncr)r%r3)rs rrrYsEYY''  .. (SYY'1 ( ..  % ')rDc||jd|_y|jrz|jjdk(rt j drt S|jjjdrt j drt StS|jr2|s0|jjdvrdt j d zS|j}|jrz|s&|jd vrdt j d zS|r-t j d st|r d|_tSt|j||rtSt St|rA|s|jjd dk(}|js|j r|st|j||rt#|dz }|dkDr||dz j sn |dz}|dkDr||dz jjdk(r#||jj%tn'|j%dt j d z|jd|_tSt|jr$|r"t j d s d|_tS|j r)|j&|jjd z }n |j&}|j&} |j(r| |jjd z } || z dkDrt StS)a2Calculate the number of newlines we need to add. Arguments: first_token: (format_token.FormatToken) The first token in the logical line. indent_depth: (int) The line's indentation depth. prev_line: (list of logical_line.LogicalLine) The logical line previous to this line. final_lines: (list of logical_line.LogicalLine) The logical lines that have already been processed. first_nested: (boolean) Whether this is the first nested class or function. Returns: The number of newlines needed before the first token. Nrr!BLANK_LINE_BEFORE_CLASS_DOCSTRINGr"BLANK_LINE_BEFORE_MODULE_DOCSTRING>fromimportrQ3BLANK_LINES_BETWEEN_TOP_LEVEL_IMPORTS_AND_VARIABLES>rrr'BLANK_LINES_AROUND_TOP_LEVEL_DEFINITION%BLANK_LINE_BEFORE_NESTED_CLASS_OR_DEFrOr)newlines is_docstringrr%r rrfrNO_BLANK_LINESis_namer'r_NoBlankLinesBeforeCurrentTokenr[rTr$rr\r^rSrW) r@rr=r<rprev_last_tokenis_inline_commentrrfirst_token_linenoprev_last_token_linenos rrr`s&'!k ( 56  //   * *3 / ))8 9   22 ?AAANN/!! [..2KKDE EE II= >k"!k &'<'$> *?+@+@++: <k"Q&% uqy)44 QJE  # ) ) / /3 6   $ $ 9 9. I  0 0EIIGHH J  ! ! -#'K   y ' eii 13# $++k.?.?.E.Ed.KK$++*11((o3399$??0014  rDc#Kd}d}|t|kr||jr||}|dz }|t|kr|jjdz}||j|jk7rn|jj dk7rlt jtjdd|j|ff}|jtj|tj|||jD]}|j||dz }|t|kr|nt!j"||d |rt||dz}|jD]}||j|t|jdk(r |j$j&r d ||_|||dz }d }n|||dz }d}|t|kry y w) aGenerate the lines we want to format. Arguments: lines: (list of logical_line.LogicalLine) Lines we want to format. Yields: Either a single line, if the current line cannot be merged with the succeeding line, or the next two lines merged into one line. rFrQrr;r)typer%contextNT)r\r$r'rGrSr%rLeafrSEMI AppendTokenr FormatTokenrNodeNamer*r CanMergeMultipleLinesrrW)r;rrlast_was_mergedrHrGleafr next_lines rrrs %/ E  U| 5\d qje CJ !!A% <  $++ -  99??c !::S2 V7L2MO$   &&t\-B-B4-HI K<&& C   3     CJ  j  * *5=/ J "i!!&# e   %& i 1 $)L)L $e  %L qjeo %L qjeoC E sD/G$2B.G$"G$c|j}|jr||jjdz}|js|jdnd}|j|z|dz k(S)a Determine if there are no blank lines before the current token. The previous token is a docstring or comment. The prev_token_lineno is the start of the text of that token. Counting the number of newlines in its text gives us the extent and thus where the line number of the end of the docstring or comment. After that, we just compare it to the current token's line number to see if there are blank lines between them. Arguments: text: (unicode) The text of the docstring or comment before the current token. cur_token: (format_token.FormatToken) The current token in the logical line. prev_token: (format_token.FormatToken) The previous token in the logical line. Returns: True if there is no blank line before the current token. rOrrQ)rSrr%rT)text cur_tokenrgcur_token_lineno num_newliness rrrsg&%% --d33)3)>)>D!A,   \ )-=-A AArD)FN)/r __future__rrrrolib2to3r lib2to3.pgen2r yapf.pytreer yapf.yapflibrrr r r rCr-r.r(r+r/r1r0r,r2r7r8objectr namedtuplerrr5rrrr rrfTWO_BLANK_LINESrrrrrrDrrs( $.%$!C0L?  "2J:66L 9 G2~B(,""6)+(()9Iw;OP$[ # #K%6 $EG 5p@G& !@)l^-`BrD