Current File : //usr/local/lib/python3.8/lib2to3/__pycache__/refactor.cpython-38.pyc
U
pak @ s d Z dZddlZddlZddlZddlZddlZddlZddlZddl m
Z
ddl m Z m
Z
mZ ddlmZ ddlmZmZ dd lmZ d!d d ZG d
d deZdd Zdd Zdd Zdd Z dd Z G dd deZ G dd d e Z G d d d eZ!G d d d e Z"dS )"zRefactoring framework.
Used as a main program, this can refactor any number of files and/or
recursively descend down directories. Imported as a module, this
provides infrastructure to write your own refactoring tool.
z#Guido van Rossum <guido@python.org> N)chain )drivertokenizetoken) find_root)pytreepygram) btm_matcherTc C sT t | g g dg}g }t|jD ].\}}}|dr |rD|dd }|| q |S )zEReturn a sorted list of all available fix names in the given package.*fix_ N)
__import__pkgutilZ iter_modules__path__
startswithappend)Z fixer_pkgZ
remove_prefixZpkgZ fix_namesfindernameZispkg r ,/usr/local/lib/python3.8/lib2to3/refactor.pyget_all_fix_names s
r c @ s e Zd ZdS )
_EveryNodeN__name__
__module__ __qualname__r r r r r + s r c C s t | tjtjfr(| jdkr t| jhS t | tjrH| jrDt| jS tt | tj rt
}| jD ] }|D ]}| t| qhq`|S t d| dS )zf Accepts a pytree Pattern Node and returns a set
of the pattern types which will match first. Nz$Oh no! I don't understand pattern %s)
isinstancer Z NodePatternZ LeafPatterntyper ZNegatedPatternZcontent_get_head_typesZWildcardPatternsetupdate Exception)patrpxr r r r / s
r c C s t t}g }| D ]x}|jrdzt|j}W n tk
rH || Y qX |D ]}|| | qNq|jdk r||j | q|| qtt j
j t j
j
D ]}|| | qt|S )z^ Accepts a list of fixers and returns a dictionary
of head node type --> fixer list. N) collections defaultdictlistpatternr r r Z _accept_typer r python_grammarZ
symbol2numbervaluestokensextenddict)Z
fixer_listZ
head_nodesZeveryfixerZheadsZ node_typer r r _get_headnode_dictK s$
r1 c s fddt dD S )zN
Return the fully qualified names for fixers in the package pkg_name.
c s g | ]} d | qS .r ).0fix_nameZpkg_namer r
<listcomp>h s z+get_fixers_from_package.<locals>.<listcomp>F)r r6 r r6 r get_fixers_from_packaged s
r8 c C s | S Nr )objr r r _identityk s r; c sX d}t t| j fdd}ttjt jtj h}t
}z| \}}||krTq>q>|tj krl|rfq6d}q>|tj kr6|dkr6| \}}|tj ks|dkrq6| \}}|tj ks|dkrq6| \}}|tj
kr|dkr| \}}|tj kr4|| | \}}|tj
ks.|d kr"q4| \}}qq>q6q>W n tk
rN Y nX t|S )
NFc s t } | d | d fS )Nr r )next)tokgenr r advancer s z(_detect_future_features.<locals>.advanceTfromZ
__future__import(,)r generate_tokensioStringIOreadline frozensetr NEWLINENLCOMMENTr STRINGNAMEOPadd
StopIteration)sourceZhave_docstringr@ ignorefeaturestpvaluer r> r _detect_future_featureso sB
rW c @ s e Zd ZdZdS )
FixerErrorz A fixer could not be loaded.N)r r r __doc__r r r r rX s rX c @ s e Zd ZdddZdZdZd4ddZdd Zd
d Zd d
Z dd Z
dd Z d5ddZ d6ddZ
dd Zd7ddZdd Zd8d d Zd d Zd d! Zd9d"d#Zd:d$d%Zd&Zd'Zd(d) Zd*d+ Zd,d- Zd.d/ Zd0d1 Z d2d3 Z dS );RefactoringToolF)print_functionwrite_unchanged_filesZFixr Nc C s. || _ |p g | _| j | _|dk r0| j| | jd rDtj| _ntj | _| j
d| _ g | _ t
d| _g | _d| _tj| jtj| jd| _| \| _| _g | _t | _ g | _ g | _ t | j| jD ]F}|j r| j !| q|| jkr| j "| q|| jkr| j "| qt#| j | _$t#| j | _%dS )zInitializer.
Args:
fixer_names: a list of fixers to import
options: a dict with configuration.
explicit: a list of fixers to run even if they are explicit.
Nr[ r\ rZ F)convertlogger)&fixersexplicit_default_optionscopyoptionsr! r !python_grammar_no_print_statementgrammarr+ getr\ errorslogging getLoggerr^ fixer_logwroter ZDriverr r]
get_fixers pre_order
post_orderfilesbmZ
BottomMatcherBMZ
bmi_pre_orderZbmi_post_orderr Z
BM_compatibleZ add_fixerr r1 bmi_pre_order_headsbmi_post_order_heads)selfZ fixer_namesrc r` r0 r r r __init__ s>
zRefactoringTool.__init__c C sX g }g }| j D ] }t|i i dg}|ddd }|| jrR|t| jd }|d}| jddd |D }zt ||}W n& t
k
r t d
||f dY nX || j | j
} | jr| jd k r|| jkr| d | q| d
| | jdkr|| q| jdkr || qt d| j qtd}
|j|
d |j|
d ||fS )a Inspects the options to load the requested patterns and handlers.
Returns:
(pre_order, post_order), where pre_order is the list of fixers that
want a pre-order AST traversal, and post_order is the list that want
post-order traversal.
r r3 r N_ c S s g | ] }| qS r )title)r4 r% r r r r7 s z.RefactoringTool.get_fixers.<locals>.<listcomp>zCan't find %s.%sTzSkipping optional fixer: %szAdding transformation: %spreZpostzIllegal fixer order: %rZ run_orderkey)r_ r rsplitr FILE_PREFIXlensplit CLASS_PREFIXjoingetattrAttributeErrorrX rc rj r` log_message log_debugorderr operator
attrgettersort) rt Zpre_order_fixersZpost_order_fixersZ fix_mod_pathmodr5 parts
class_nameZ fix_classr0 Zkey_funcr r r rl s:
zRefactoringTool.get_fixersc O s dS )z Called when an error occurs.Nr )rt msgargskwdsr r r log_error s zRefactoringTool.log_errorc G s |r || }| j | dS )zHook to log a message.N)r^ infort r r r r r r s zRefactoringTool.log_messagec G s |r || }| j | d S r9 )r^ debugr r r r r s zRefactoringTool.log_debugc C s dS )zTCalled with the old version, new version, and filename of a
refactored file.Nr )rt old_textnew_textfilenameequalr r r print_output s z RefactoringTool.print_outputc C s8 |D ].}t j|r$| ||| q| ||| qdS )z)Refactor a list of files and directories.N)ospathisdir refactor_dir
refactor_file)rt itemswrite
doctests_onlyZ dir_or_filer r r refactor s zRefactoringTool.refactorc
C s t jd }t |D ]\}}}| d| | | |D ]>}|ds>t j|d |kr>t j||} | | || q>dd |D |dd< qdS )zDescends down a directory and refactor every Python file found.
Python files are assumed to have a .py extension.
Files and subdirectories starting with '.' are skipped.
pyzDescending into %sr3 r c S s g | ]}| d s|qS r2 )r )r4 Zdnr r r r7 . s
z0RefactoringTool.refactor_dir.<locals>.<listcomp>N)
r extsepwalkr r r r splitextr r )
rt Zdir_namer r Zpy_extdirpathdirnames filenamesr fullnamer r r r s
z RefactoringTool.refactor_dirc
C s zt |d}W n6 tk
rD } z| d|| W Y dS d}~X Y nX zt|jd }W 5 | X tj |d|dd}| |fW 5 Q R S Q R X dS ) zG
Do our best to decode a Python source file correctly.
rbzCan't open %s: %s)NNNr r$ rx encodingnewline) openOSErrorr closer detect_encodingrH rF read)rt r ferrr r r r _read_python_source0 s
z#RefactoringTool._read_python_sourcec C s | |\}}|dkrdS |d7 }|rn| d| | ||}| jsL||kr`| ||||| q| d| nH| ||}| js|r|jr| jt|dd |||d n | d| dS )zRefactors a file.N
zRefactoring doctests in %szNo doctest changes in %srv )r r zNo changes in %s)r r refactor_docstringr\ processed_filerefactor_string was_changedstr)rt r r r inputr outputtreer r r r @ s" z RefactoringTool.refactor_filec
C s t |}d|krtj| j_zVz| j|}W n@ tk
rl } z"| d||jj | W Y W dS d}~X Y nX W 5 | j| j_X ||_
| d| | || |S )aF Refactor a given input string.
Args:
data: a string holding the code to be refactored.
name: a human-readable name for use in error/log messages.
Returns:
An AST corresponding to the refactored input stream; None if
there were errors during the parse.
r[ zCan't parse %s: %s: %sNzRefactoring %s)
rW r rd r re Z parse_stringr" r __class__r future_featuresr
refactor_tree)rt datar rT r r r r r r W s"
z RefactoringTool.refactor_stringc C s t j }|rN| d | |d}| js2||krB| |d| q| d n:| |d}| jsj|r~|jr~| t |d| n
| d d S )Nz Refactoring doctests in stdinz<stdin>zNo doctest changes in stdinzNo changes in stdin)
sysstdinr r r r\ r r r r )rt r r r r r r r refactor_stdinr s
z RefactoringTool.refactor_stdinc
C s t | j| jD ]}||| q| | j| | | j| | j| }t
| r| jj D ]D}||krj|| rj|| j
tjjdd |jr|| j
tjjd t|| D ]}||| kr|| | z t| W n tk
r Y qY nX |jr||jkrq||}|r|||}|dk r|| | D ] }|js^g |_|j| qL| j| }|D ]*} | |krg || < || || qqqjqTt | j| jD ]}| || q|j S )a Refactors a parse tree (modifying the tree in place).
For compatible patterns the bottom matcher module is
used. Otherwise the tree is traversed node-to-node for
matches.
Args:
tree: a pytree.Node instance representing the root of the tree
to be refactored.
name: a human-readable name for this tree.
Returns:
True if the tree was modified, False otherwise.
T)r| reverser{ N) r rm rn Z
start_tree traverse_byrr rs rq runZleavesanyr, r_ r r ZBasedepthZkeep_line_orderZ
get_linenor) remover
ValueErrorZfixers_appliedmatch transformreplacer r. Z finish_treer )
rt r r r0 Z match_setnoderesultsnewZ new_matchesZfxrr r r r sJ
z RefactoringTool.refactor_treec C sV |sdS |D ]D}||j D ]4}||}|r|||}|dk r|| |}qq dS )a Traverse an AST, applying a set of fixers to each node.
This is a helper method for refactor_tree().
Args:
fixers: a list of fixer instances.
traversal: a generator that yields AST nodes.
Returns:
None
N)r r r r )rt r_ Z traversalr r0 r r r r r r s
zRefactoringTool.traverse_byc C s | j | |dkr.| |d }|dkr.dS ||k}| |||| |r`| d| | js`dS |rv| |||| n | d| dS )zR
Called when a file has been refactored and there may be changes.
Nr zNo changes to %szNot writing changes to %s)ro r r r r r\
write_file)rt r r r r r r r r r r s z RefactoringTool.processed_filec C s zt j|d|dd}W n6 tk
rL } z| d|| W Y dS d}~X Y nX |F z|| W n0 tk
r } z| d|| W 5 d}~X Y nX W 5 Q R X | d| d| _dS ) zWrites a string to a file.
It first shows a unified diff between the old text and the new text, and
then rewrites the file; the latter is only done if the write option is
set.
wrx r zCan't create %s: %sNzCan't write %s: %szWrote changes to %sT)rF r r r r r rk )rt r r r r fpr r r r r s * zRefactoringTool.write_filez>>> z... c
C s g }d}d}d}d}|j ddD ]}|d7 }| | jr~|dk rZ|| |||| |}|g}|| j} |d| }q |dk r||| j s||| j d kr| | q |dk r|| |||| d}d}| | q |dk r
|| |||| d
|S )a Refactors a docstring, looking for doctests.
This returns a modified version of the input string. It looks
for doctests, which start with a ">>>" prompt, and may be
continued with "..." prompts, as long as the "..." is indented
the same as the ">>>".
(Unfortunately we can't use the doctest module's parser,
since, like most parsers, it is not geared towards preserving
the original source.)
Nr Tkeependsr r rx )
splitlineslstripr PS1r. refactor_doctestfindPS2rstripr r )
rt r r resultblockZ block_linenoindentlinenolineir r r r sJ
z"RefactoringTool.refactor_docstringc
s. z || }W nj tk
r| } zLjtjrN|D ]}d|d q6d|||j j
| | W Y S d}~X Y nX ||r*t |j
dd}|d|d ||d d } }| dg|d kst| |d ds|d d7 < j |d g}|r*| fd
d |D 7 }|S ) zRefactors one doctest.
A doctest is given as a block of lines, the first of which starts
with ">>>" (possibly indented), while the remaining lines start
with "..." (identically indented).
z
Source: %sr z+Can't parse docstring in %s line %s: %s: %sNTr r rv r c s g | ]} j | qS r )r )r4 r r rt r r r7 Z s z4RefactoringTool.refactor_doctest.<locals>.<listcomp>) parse_blockr" r^ isEnabledForrh DEBUGr r r r r r r r AssertionErrorendswithr pop)
rt r r r r r r r r Zclippedr r r r @ s. "z RefactoringTool.refactor_doctestc C s | j r d}nd}| js$| d| n"| d| | jD ]}| | q6| jrl| d | jD ]}| | q\| jrt| jdkr| d n| dt| j | jD ] \}}}| j|f|| qd S ) Nwerez
need to bezNo files %s modified.zFiles that %s modified:z$Warnings/messages while refactoring:r zThere was 1 error:zThere were %d errors:)rk ro r rj rg r )rt r filemessager r r r r r summarize] s$
zRefactoringTool.summarizec C s" | j | |||}t |_|S )zParses a block into a tree.
This is necessary to get correct line number / offset information
in the parser diagnostics and embedded into the parse tree.
)r Z parse_tokens wrap_toksrI r )rt r r r r r r r r t s zRefactoringTool.parse_blockc c sd t | ||j}|D ]F\}}\}}\} }
} ||d 7 }| |d 7 } ||||f| |
f| fV qdS )z;Wraps a tokenize stream to systematically modify start/end.r N)r rE gen_lines__next__) rt r r r r- r rV Zline0Zcol0Zline1Zcol1Z line_textr r r r ~ s
zRefactoringTool.wrap_toksc c sx || j }|| j }|}|D ]N}||r>|t|d V n(|| d krVdV ntd||f |}q dV qldS )zGenerates lines as expected by tokenize from a list of lines.
This strips the first len(indent + self.PS1) characters off each line.
Nr zline=%r, prefix=%rrx )r r r r r r )rt r r prefix1Zprefix2prefixr r r r r s
zRefactoringTool.gen_lines)NN)FF)FF)FF)F)NFN)N) r r r ra r r~ ru rl r r r r r r r r r r r r r r r r r r r r r r r r r r rZ s>
4(
O
+
rZ c @ s e Zd ZdS )MultiprocessingUnsupportedNr r r r r r s r c sB e Zd Z fddZd fdd Z fddZ fd d
Z ZS ) MultiprocessRefactoringToolc s" t t| j|| d | _d | _d S r9 )superr ru queue output_lockrt r kwargsr r r ru s z$MultiprocessRefactoringTool.__init__Fr c
s |dkr t t|||S z dd l W n tk
r@ tY nX jd k rTtd _ _
fddt |D }z*|D ] }| qt t||| W 5 j t |D ]}j
d q|D ]}| r| qd _X d S )Nr r z already doing multiple processesc s g | ]} j jd qS ))target)Process_child)r4 r multiprocessingrt r r r7 s z8MultiprocessRefactoringTool.refactor.<locals>.<listcomp>)r r r r ImportErrorr r RuntimeError
JoinableQueueLockr ranger putis_alivestart)rt r r r Z
num_processes processesr r% r r r r s<
z$MultiprocessRefactoringTool.refactorc sN | j }|d k rJ|\}}ztt| j|| W 5 | j X | j }q
d S r9 )r rf task_doner r r )rt Ztaskr r r r r r s
z"MultiprocessRefactoringTool._childc s2 | j d k r | j ||f ntt| j||S d S r9 )r r r r r r r r r r s
z)MultiprocessRefactoringTool.refactor_file)FFr )r r r ru r r r
__classcell__r r r r r s r )T)#rY
__author__rF r r r rh r r' itertoolsr Zpgen2r r r Z
fixer_utilr rx r r r
rp r r" r r r1 r8 r; rW rX objectrZ r r r r r r <module> s8
(
Mini Shell