xfddlZddlZddlmZmZmZmZmZm Z ddlm Z m Z m Z ddl ZddlZddlmZddlZddlmZddlmZmZgdZGd d eZd Zd Zd ZdZedjAdjAZ!dZ" d8dZ#e"e# d9dZ$GddZ%GddZ&GddZ'dZ(Gdde&Z)GddZ*d jAe!d!<Gd"d#e)Z+Gd$d%e+Z,Gd&d'e)Z-Gd(d)e)Z.Gd*d+e)Z/Gd,d-e)Z0Gd.d/e&Z1d0Z2e2d1e+Z3e2d2e,Z4e2d3e-Z5e2d4e/Z6e2d5e.Z7e2d6e0Z8e2d7e1Z9y):N)normsolveinvqrsvd LinAlgError)asarraydotvdot)get_blas_funcs)getfullargspec_no_self)scalar_search_wolfe1scalar_search_armijo) broyden1broyden2anderson linearmixing diagbroydenexcitingmixing newton_krylov BroydenFirstKrylovJacobianInverseJacobianc eZdZy) NoConvergenceN)__name__ __module__ __qualname__8/usr/lib/python3/dist-packages/scipy/optimize/_nonlin.pyrrsr!rcHtj|jSN)npabsolutemaxxs r"maxnormr*s ;;q>   r!ct|}tj|jtjst|tj S|S)z:Return `x` as an array, of either floats or complex floatsdtype)r r% issubdtyper-inexactfloat_r(s r" _as_inexactr1"s7 A =="** -q ** Hr!ctj|tj|}t|d|j}||S)z;Return ndarray `x` as same array subclass and shape as `x0`__array_wrap__)r%reshapeshapegetattrr3)r)x0wraps r" _array_liker9*s8 1bhhrl#A 2')9)9 :D 7Nr!ctj|js#tjtjSt |Sr$)r%isfiniteallarrayinfr)vs r" _safe_normr@1s2 ;;q>   xx 7Nr!z F : function(x) -> f Function whose root to find; should take and return an array-like object. xin : array_like Initial guess for the solution a iter : int, optional Number of iterations to make. If omitted (default), make as many as required to meet tolerances. verbose : bool, optional Print status to stdout on every iteration. maxiter : int, optional Maximum number of iterations to make. If more are needed to meet convergence, `NoConvergence` is raised. f_tol : float, optional Absolute tolerance (in max-norm) for the residual. If omitted, default is 6e-6. f_rtol : float, optional Relative tolerance for the residual. If omitted, not used. x_tol : float, optional Absolute minimum step size, as determined from the Jacobian approximation. If the step size is smaller than this, optimization is terminated as successful. If omitted, not used. x_rtol : float, optional Relative minimum step size. If omitted, not used. tol_norm : function(vector) -> scalar, optional Norm to use in convergence check. Default is the maximum norm. line_search : {None, 'armijo' (default), 'wolfe'}, optional Which type of a line search to use to determine the step size in the direction given by the Jacobian approximation. Defaults to 'armijo'. callback : function, optional Optional callback function. It is called on every iteration as ``callback(x, f)`` where `x` is the current solution and `f` the corresponding residual. Returns ------- sol : ndarray An array (of similar array type as `x0`) containing the final solution. Raises ------ NoConvergence When a solution was not found. ) params_basic params_extracN|jr|jtz|_yyr$)__doc__ _doc_parts)objs r"_set_docrGos {{kkJ. r!c | tn| } t|||| || }tfd}j}t j |tj }||}t|}t|}|j|j|||||dz}nd|jdzz}| durd} n| durd} | d vr td d }d }d }d}t|D]C}|j|||}|rnDt|||z}|j!|| }t|dk(r td| rt#||||| \}}}}nd}||z}||}t|}|j%|j|| r | ||||dzz|dzz }||dzz|kr t||}nt|t'|||dzz}|}|st(j*j-d|| ||fzt(j*j/F|rt1t3|d}| r)|j4|||dk(ddd|d}t3||fSt3|S)a Find a root of a function, in a way suitable for large-scale problems. Parameters ---------- %(params_basic)s jacobian : Jacobian A Jacobian approximation: `Jacobian` object or something that `asjacobian` can transform to one. Alternatively, a string specifying which of the builtin Jacobian approximations to use: krylov, broyden1, broyden2, anderson diagbroyden, linearmixing, excitingmixing %(params_extra)s full_output : bool If true, returns a dictionary `info` containing convergence information. raise_exception : bool If True, a `NoConvergence` exception is raise if no solution is found. See Also -------- asjacobian, Jacobian Notes ----- This algorithm implements the inexact Newton method, with backtracking or full line searches. Several Jacobian approximations are available, including Krylov and Quasi-Newton methods. References ---------- .. [KIM] C. T. Kelley, "Iterative Methods for Linear and Nonlinear Equations". Society for Industrial and Applied Mathematics. (1995) https://archive.siam.org/books/kelley/fr16/ N)f_tolf_rtolx_tolx_rtoliterrcVtt|jSr$)r1r9flatten)zFr7s r"funcznonlin_solve..funcs#1[B/0199;;r!rdTarmijoF)NrTwolfezInvalid line searchg?gH.?g?gMbP?)tolrz[Jacobian inversion yielded zero vector. This indicates a bug in the Jacobian approximation.?z%d: |F(x)| = %g; step %g z0A solution was found at the specified tolerance.z:The maximum number of iterations allowed has been reached.)rrX)nitfunstatussuccessmessage)r*TerminationConditionr1rOr% full_liker>r asjacobiansetupcopysize ValueErrorrangecheckminr_nonlin_line_searchupdater'sysstdoutwriteflushrr9 iteration) rQr7jacobianrMverbosemaxiterrIrJrKrLtol_norm line_searchcallback full_outputraise_exception conditionrRr)dxFxFx_normgammaeta_max eta_tresholdetanr[rVs Fx_norm_neweta_Ainfos `` r" nonlin_solvertsZ#*wH$5+0*.X?I RB< A a B aB2hG(#H NN1668R&  QhG166!8nGd   33.// EGL C 7^/Q+  #s7{#nnRSn) ) 8q=./ / $7aR8C%E !Aq"kABAaBr(K"%  QO Q&!3 36>L (gu%Cgs5%Q,78C  JJ  :8B<>$$ % JJ   U/X  Ar 23 3F ** !Q; , 3% & 1b!4''1b!!r!cl dg|gt|dzgttz d fd  fd}|dk(rt |dd|\}} } n|dk(rt dd | \}} d }|zz|dk(rd}n}t|} ||| fS) NrrXc| dk(rdS |zz}|}t|dz}|r| d<|d<|d<|S)NrrX)r@) rstorextr?prxrRtmp_Fxtmp_phitmp_sr)s r"phiz _nonlin_line_search..phis_ a=1:  2X H qM1  E!HGAJF1Ir!c^t|zdzz}||zd|z |z S)NrF)r)abs)rdsrrdiffs_norms r"derphiz#_nonlin_line_search..derphis9!fvo!U *AbD&Q/255r!rU{Gz?)xtolaminrT)rrW)T)rrr)rRr)ryrx search_typersminrrphi1phi0rzrrrrrs`` ` ` @@@@@r"rhrh s CETFBx{mG !WtBx F  6g,S&'!*26TC 4  &sGAJ ,024 y  AbDAE!H} AY !W2hG aW r!c*eZdZdZdddddefdZdZy)r^z Termination condition for an iteration. It is terminated if - |F| < f_rtol*|F_0|, AND - |F| < f_tol AND - |dx| < x_rtol*|x|, AND - |dx| < x_tol NcD|0tjtjjdz}|tj}|tj}|tj}||_||_||_||_||_ ||_ d|_ d|_ y)NgUUUUUU?r) r%finfor0epsr>rKrLrIrJrrMf0_normrn)selfrIrJrKrLrMrs r"__init__zTerminationCondition.__init__Ds =HHRYY'++5E >VVF =FFE >VVF       r!c|xjdz c_|j|}|j|}|j|}|j||_|dk(ry|jd|j|jkDzSt ||j kxr||j z |jkxr#||jkxr||jz |kS)NrrrX) rnrrrMintrIrJrKrL)rfr)rxf_normx_normdx_norms r"rfzTerminationCondition.check\s !11))B- << !DL Q; 99 23 3Fdjj(;t{{*dll:;4::-:#DKK/69< z#Jacobian.__init__..s T\\^r!)itemsrdsetattrhasattr __array__)rkwnamesnamevalues` r"rzJacobian.__init__sj888: .KD%5  !>!EFF dBtH-  . 4 #3DN $r!ct|Sr$)rrs r"aspreconditionerzJacobian.aspreconditioners t$$r!ctr$NotImplementedErrorrr?rVs r"rzJacobian.solve!!r!cyr$r rr)rQs r"rizJacobian.update r!c||_|j|jf|_|j|_|jj t j ur|j||yyr$)rRrcr5r- __class__rarrirr)rQrRs r"razJacobian.setupsQ ffaff% WW >>  8>> 1 KK1  2r!Nr) rrrrDrrrrirar r!r"rrws!#J 4%" r!rc2eZdZdZedZedZy)rc||_|j|_|j|_t |dr|j |_t |dr|j |_yy)Nrar)rorrrirrarr)rros r"rzInverseJacobian.__init__sN  nn oo 8W %!DJ 8X &#??DL 'r!c.|jjSr$)ror5rs r"r5zInverseJacobian.shape}}"""r!c.|jjSr$)ror-rs r"r-zInverseJacobian.dtyperr!N)rrrrpropertyr5r-r r!r"rrs/+####r!rc (tjjjt t rSt jrtt rSt tjrjdkDr tdtjtjjdjdk7r tdt fdfdfdfd j j Stjj#r]jdjdk7r td t fd fd fdfdj j St%dr{t%drot%drct t'dt'dj(t'dt'dt'dj jSt+rGfddt }|St t,r6t/t0t2t4t6t8t:t<St?d)zE Convert given object to one suitable for use as a Jacobian. rXzarray must have rank <= 2rrzarray must be squarect|Sr$)r r?Js r"rzasjacobian..s Qr!cLtjj|Sr$)r conjTrs r"rzasjacobian..s#affhjj!*<r!ct|Sr$)rrs r"rzasjacobian..s a r!cLtjj|Sr$)rrrrs r"rzasjacobian..sqvvxzz1)=r!)rrrrr-r5zmatrix must be squarec|zSr$r rs r"rzasjacobian..s 1r!c>jj|zSr$rrrs r"rzasjacobian..s!&&(**q.r!c|Sr$r r?rspsolves r"rzasjacobian..s 1 r!cFjj|Sr$rrs r"rzasjacobian..sQ)?r!r5r-rrrrrira)rrrrrirar-r5cDeZdZdZdfd ZfdZdfd ZfdZy)asjacobian..Jacc||_yr$r(rs r"rizasjacobian..Jac.updates r!c|j}t|tjr t ||St j j|r ||StdNzUnknown matrix type) r) isinstancer%ndarrayrscipysparse isspmatrixrdrr?rVmrrs r"rzasjacobian..Jac.solvesTdffIa, A;&\\,,Q/"1a=($%:;;r!c|j}t|tjr t ||St j j|r||zStdr) r)rr%rr rrrrdrr?rrs r"rzasjacobian..Jac.matvecsQdffIa,q!9$\\,,Q/Q3J$%:;;r!c:|j}t|tjr$t |j j |Stjj|r!|j j |Stdr) r)rr%rrrrrrrrdrs r"rzasjacobian..Jac.rsolve sjdffIa, Q//\\,,Q/"1668::q11$%:;;r!c2|j}t|tjr$t |j j |Stjj|r|j j |zStdr) r)rr%rr rrrrrrdrs r"rzasjacobian..Jac.rmatvecsgdffIa,qvvxzz1--\\,,Q/668::>)$%:;;r!Nr)rrrrirrrr)rrsr"Jacrs  < < < >r!ceZdZdZdZdZy)GenericBroydenctj||||||_||_t |drC|j 6t |}|r!dtt |dz|z |_yd|_yyy)Nalpha?rrW)rralast_flast_xrrrr')rr7f0rRnormf0s r"razGenericBroyden.setup/sntRT*  4 !djj&8"XF T"Xq!11F:   '9 !r!ctr$rrr)rrxdfrdf_norms r"_updatezGenericBroyden._update=rr!c ||jz }||jz }|j||||t|t|||_||_yr$)rrr r)rr)rr rxs r"rizGenericBroyden.update@sH _ _ Q2r48T"X6  r!N)rrrrar rir r!r"rr.s !"r!rcxeZdZdZdZedZedZdZdZ ddZ ddZ d Z d Z d Zd Zd ZddZy) LowRankMatrixz A matrix represented as .. math:: \alpha I + \sum_{n=0}^{n=M} c_n d_n^\dagger However, if the rank of the matrix reaches the dimension of the vectors, full matrix representation will be used thereon. cX||_g|_g|_||_||_d|_yr$)rcsrrr- collapsed)rrrr-s r"rzLowRankMatrix.__init__Ss,  r!ctgd|dd|gz\}}}||z}t||D]#\}} || |} ||||j| }%|S)N)axpyscaldotcr)r ziprc) r?rrrrrrwcdas r"_matveczLowRankMatrix._matvec[sm)*B*,Ra&A3,8dD AIBK &DAqQ AQ1661%A &r!c Jt|dk(r||z Stddg|dd|gz\}}|d}|tjt||jz}t |D].\}} t |D]\} } ||| fxx|| | z cc<0tj t||j} t |D]\} } || || | <| |z} t|| } ||z } t|| D]\} }|| | | j| } | S)Evaluate w = M^-1 vrrrNrr,) lenr r%identityr- enumeratezerosrrrc)r?rrrrrc0Airjrqrqcs r"_solvezLowRankMatrix._solvees: r7a<U7N$VV$4b!fslC d U BKKBrxx8 8bM %DAq!"  %1!A#$q!*$ % % HHSWBHH -bM DAq1:AaD  U  !QK eGQZ (EArQ166B3'A (r!c|j tj|j|Stj ||j |j |jS)zEvaluate w = M v)rr%r rrrrrrr?s r"rzLowRankMatrix.matvecsD >> %66$..!, ,$$Q DGGTWWEEr!c|j8tj|jjj |St j |tj|j|j|jS)zEvaluate w = M^H v) rr%r rrrrrrrr,s r"rzLowRankMatrix.rmatvecs\ >> %66$..**//115 5$$Q (;TWWdggNNr!c|jt|j|Stj||j|j |j S)r)rrrr*rrrrs r"rzLowRankMatrix.solves@ >> %+ +##Atzz477DGGDDr!c|j.t|jjj|Stj |t j|j|j|jS)zEvaluate w = M^-H v) rrrrrr*r%rrrrs r"rzLowRankMatrix.rsolvesX >> %))..0!4 4##Arwwtzz':DGGTWWMMr!cX|j5|xj|dddf|dddfjzz c_y|jj||jj|t |j|j kDr|jyyr$)rrrappendrr rccollapse)rrrs r"r1zLowRankMatrix.appends{ >> % NNa$i!DF)..*:: :N  q q tww|j |jS|jtj|j|j z}t |j|jD])\}}||dddf|dddfjzz }+|S)Nr,) rrr%r!rr-rrrr)rGmrrs r"rzLowRankMatrix.__array__s >> %>> ! ZZ DFF$**= =) -DAq !AdF)Ad1fINN,, ,B - r!cbtj||_d|_d|_d|_y)z0Collapse the low-rank matrix to a full-rank one.N)r%r=rrrrrs r"r2zLowRankMatrix.collapses&$ r!c|jy|dkDsJt|j|kDr|jdd=|jdd=yy)zH Reduce the rank of the matrix by dropping all vectors. Nrrr rrrranks r"restart_reducezLowRankMatrix.restart_reducesG >> % axx tww<$    r!c|jy|dkDsJt|j|kDr4|jd=|jd=t|j|kDr3yy)zK Reduce the rank of the matrix by dropping oldest vectors. Nrr7r8s r" simple_reducezLowRankMatrix.simple_reducesT >> % axx$''lT!  $''lT!r!Nc|jy|}||}n|dz }|jr"t|t|jd}t dt||dz }t|j}||kryt j |jj}t j |jj}t|d\}}t||jj}t|d\} } } t|t| }t|| jj}t|D]J} |dd| fj|j| <|dd| fj|j| <L|j|d=|j|d=y) a Reduce the rank of the matrix by retaining some SVD components. This corresponds to the "Broyden Rank Reduction Inverse" algorithm described in [1]_. Note that the SVD decomposition can be done by solving only a problem whose size is the effective rank of this matrix, which is viable even for large problems. Parameters ---------- max_rank : int Maximum rank of this matrix after reduction. to_retain : int, optional Number of SVD components to retain when reduction is done (ie. rank > max_rank). Default is ``max_rank - 2``. References ---------- .. [1] B.A. van der Rotten, PhD thesis, "A limited memory Broyden method to solve high-dimensional systems of nonlinear equations". Mathematisch Instituut, Universiteit Leiden, The Netherlands (2003). https://web.archive.org/web/20161022015821/http://www.math.leidenuniv.nl/scripties/Rotten.pdf NrXrreconomic)modeF) full_matrices)rrrgr r'r%r=rrrr rrrrerb) rmax_rank to_retainrr(rCDRUSWHks r" svd_reducezLowRankMatrix.svd_reducesh: >> %    AAA 77As4771:'A 3q!A#;  L q5  HHTWW    HHTWW   !*%1 13388: q.1b 3r7O 24499; q 'A1Q3DGGAJ1Q3DGGAJ ' GGABK GGABKr!rr$)rrrrDr staticmethodrr*rrrrr1rr2r:r<rJr r!r"rrHsj6F O E N   ?r!ra alpha : float, optional Initial guess for the Jacobian is ``(-1/alpha)``. reduction_method : str or tuple, optional Method used in ensuring that the rank of the Broyden matrix stays low. Can either be a string giving the name of the method, or a tuple of the form ``(method, param1, param2, ...)`` that gives the name of the method and values for additional parameters. Methods available: - ``restart``: drop all matrix columns. Has no extra parameters. - ``simple``: drop oldest matrix column. Has no extra parameters. - ``svd``: keep only the most significant SVD components. Takes an extra parameter, ``to_retain``, which determines the number of SVD components to retain when rank reduction is done. Default is ``max_rank - 2``. max_rank : int, optional Maximum rank for the Broyden matrix. Default is infinity (i.e., no rank reduction). broyden_paramscFeZdZdZd dZdZdZd dZdZd dZ d Z d Z y) ra Find a root of a function, using Broyden's first Jacobian approximation. This method is also known as \"Broyden's good method\". Parameters ---------- %(params_basic)s %(broyden_params)s %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='broyden1'`` in particular. Notes ----- This algorithm implements the inverse Jacobian Quasi-Newton update .. math:: H_+ = H + (dx - H df) dx^\dagger H / ( dx^\dagger H df) which corresponds to Broyden's first Jacobian update .. math:: J_+ = J + (df - J dx) dx^\dagger / dx^\dagger dx References ---------- .. [1] B.A. van der Rotten, PhD thesis, \"A limited memory Broyden method to solve high-dimensional systems of nonlinear equations\". Mathematisch Instituut, Universiteit Leiden, The Netherlands (2003). https://web.archive.org/web/20161022015821/http://www.math.leidenuniv.nl/scripties/Rotten.pdf Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * (x[0] - x[1])**3 - 1.0, ... 0.5 * (x[1] - x[0])**3 + x[1]] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.broyden1(fun, [0, 0]) >>> sol array([0.84116396, 0.15883641]) NcLtj|_d_|tj }|_t|trdn |dd|d}|dz fz|dk(r fd_ y|dk(r fd_ y|dk(r fd _ ytd |z) Nr rrrc6jjSr$)r4rJ reduce_paramsrsr"rz'BroydenFirst.__init__..ks#5477#5#5}#Er!simplec6jjSr$)r4r<rPsr"rz'BroydenFirst.__init__..ms#8477#8#8-#Hr!restartc6jjSr$)r4r:rPsr"rz'BroydenFirst.__init__..os#9477#9#9=#Ir!z"Unknown rank reduction method '%s') rrrr4r%r>rArr_reducerd)rrreduction_methodrArQs` @r"rzBroydenFirst.__init__Zs%   vvH  & ,M,QR0M/2 !A-7 u $EDL  )HDL  *IDLA-./ /r!ctj||||t|j |jd|j |_y)Nr)rrarrr5r-r4rs r"razBroydenFirst.setupts8T1a. TZZ]DJJGr!c,t|jSr$)rr4rs r"rzBroydenFirst.todensexs477|r!c|jj|}tj|j sL|j |j |j|j|jj|S|Sr$) r4rr%r;r<rarrrR)rrrVrs r"rzBroydenFirst.solve{s\ GGNN1 {{1~!!# JJt{{DKK ;77>>!$ $r!c8|jj|Sr$)r4rrrs r"rzBroydenFirst.matvecsww}}Qr!c8|jj|Sr$)r4rrrrVs r"rzBroydenFirst.rsolveswwq!!r!c8|jj|Sr$)r4rr]s r"rzBroydenFirst.rmatvecsww~~a  r!c|j|jj|}||jj|z }|t ||z } |jj || yr$)rVr4rrr r1 rr)rrxr rr r?rrs r"r zBroydenFirst._updatesU  GGOOB  # # R O q!r!)NrTNr) rrrrDrrarrrrrr r r!r"rr$s13j/4H "!r!rceZdZdZdZy)raK Find a root of a function, using Broyden's second Jacobian approximation. This method is also known as "Broyden's bad method". Parameters ---------- %(params_basic)s %(broyden_params)s %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='broyden2'`` in particular. Notes ----- This algorithm implements the inverse Jacobian Quasi-Newton update .. math:: H_+ = H + (dx - H df) df^\dagger / ( df^\dagger df) corresponding to Broyden's second method. References ---------- .. [1] B.A. van der Rotten, PhD thesis, "A limited memory Broyden method to solve high-dimensional systems of nonlinear equations". Mathematisch Instituut, Universiteit Leiden, The Netherlands (2003). https://web.archive.org/web/20161022015821/http://www.math.leidenuniv.nl/scripties/Rotten.pdf Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * (x[0] - x[1])**3 - 1.0, ... 0.5 * (x[1] - x[0])**3 + x[1]] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.broyden2(fun, [0, 0]) >>> sol array([0.84116365, 0.15883529]) c|j|}||jj|z }||dzz } |jj|| yNrX)rVr4rr1rbs r"r zBroydenSecond._updatesF   # #  N q!r!N)rrrrDr r r!r"rrs 0dr!rc,eZdZdZddZddZdZdZy) ra Find a root of a function, using (extended) Anderson mixing. The Jacobian is formed by for a 'best' solution in the space spanned by last `M` vectors. As a result, only a MxM matrix inversions and MxN multiplications are required. [Ey]_ Parameters ---------- %(params_basic)s alpha : float, optional Initial guess for the Jacobian is (-1/alpha). M : float, optional Number of previous vectors to retain. Defaults to 5. w0 : float, optional Regularization parameter for numerical stability. Compared to unity, good values of the order of 0.01. %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='anderson'`` in particular. References ---------- .. [Ey] V. Eyert, J. Comp. Phys., 124, 271 (1996). Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * (x[0] - x[1])**3 - 1.0, ... 0.5 * (x[1] - x[0])**3 + x[1]] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.anderson(fun, [0, 0]) >>> sol array([0.84116588, 0.15883789]) Nctj|||_||_g|_g|_d|_||_yr$)rrrMrxr r{w0)rrrirhs r"rzAnderson.__init__s:%  r!c|j |z}t|j}|dk(r|Stj||j }t |D]}t|j||||<  t|j|}t |D]7}||||j||j|j|zzzz }9|S#t$r#|jdd=|jdd=|cYSwxYwNrr,) rr rxr%emptyr-rer r rrr) rrrVrxrdf_frIr{rs r"rzAnderson.solve&sjj[] L 6Ixx)q *A4771:q)DG * $&&$'Eq @A %(DGGAJDGGAJ)>>? ?B @    I  s;C)DDc B| |jz }t|j}|dk(r|Stj||j }t |D]}t|j||||< tj||f|j }t |D]}t |D]}t|j||j||||f<||k(s4|jdk7sD|||fxxt|j||j||jdzz|jzzcc<t||} t |D]7} || | |j| |j| |jz zzz }9|S)Nrr,rX) rr rxr%rlr-rer r rir) rrrxrrmrIbr&r'r{rs r"rzAnderson.matvec=syR ] L 6Ixx)q *A4771:q)DG * HHaV177 +q QA1X Qdggaj$''!*5!A#6dgglacFd4771:twwqz:477A:EdjjPPF Q Q aq @A %(DGGAJDJJ)>>? ?B @ r!c8|jdk(ry|jj||jj|t |j|jkDrY|jj d|jj dt |j|jkDrYt |j}t j||f|j}t|D][} t| |D]J} | | k(r|jdz} nd} d| zt|j| |j| z|| | f<L]|t j|djjz }||_y)Nrr,rXr)rhrxr1r r popr%r#r-rerir triurrr) rr)rrxr rr rrr&r'wds r"r zAnderson._updateTs7 66Q;  r r$''lTVV# GGKKN GGKKN$''lTVV# L HHaV177 +q =A1a[ =6!BBB$TWWQZ <<!A#  = = RWWQ]__ ! ! ##r!)Nrr)rrrrDrrrr r r!r"rrs+L..r!rcFeZdZdZd dZdZd dZdZd dZdZ d Z d Z y) ra, Find a root of a function, using diagonal Broyden Jacobian approximation. The Jacobian approximation is derived from previous iterations, by retaining only the diagonal of Broyden matrices. .. warning:: This algorithm may be useful for specific problems, but whether it will work may depend strongly on the problem. Parameters ---------- %(params_basic)s alpha : float, optional Initial guess for the Jacobian is (-1/alpha). %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='diagbroyden'`` in particular. Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * (x[0] - x[1])**3 - 1.0, ... 0.5 * (x[1] - x[0])**3 + x[1]] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.diagbroyden(fun, [0, 0]) >>> sol array([0.84116403, 0.15883384]) Nc<tj|||_yr$rrrrrs r"rzDiagBroyden.__init__% r!ctj||||tj|jdfd|j z |j |_y)Nrrr,)rrar%fullr5rr-rrs r"razDiagBroyden.setupsAT1a.$**Q-)1tzz>Lr!c"| |jz Sr$rr_s r"rzDiagBroyden.solverDFF{r!c"| |jzSr$r}r]s r"rzDiagBroyden.matvecr~r!c>| |jjz Sr$rrr_s r"rzDiagBroyden.rsolverDFFKKM!!r!c>| |jjzSr$rr]s r"rzDiagBroyden.rmatvecrr!cBtj|j Sr$)r%diagrrs r"rzDiagBroyden.todenseswwwr!c`|xj||j|zz|z|dzz zc_yrer}r s r"r zDiagBroyden._updates* 2r >2%gqj00r!r$r rrrrDrrarrrrrr r r!r"rrrs1&PM"" 1r!rc@eZdZdZd dZd dZdZd dZdZdZ d Z y) ra Find a root of a function, using a scalar Jacobian approximation. .. warning:: This algorithm may be useful for specific problems, but whether it will work may depend strongly on the problem. Parameters ---------- %(params_basic)s alpha : float, optional The Jacobian approximation is (-1/alpha). %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='linearmixing'`` in particular. Nc<tj|||_yr$rwrxs r"rzLinearMixing.__init__ryr!c"| |jzSr$rr_s r"rzLinearMixing.solver$**}r!c"| |jz Sr$rr]s r"rzLinearMixing.matvecrr!cH| tj|jzSr$r%rrr_s r"rzLinearMixing.rsolver"''$**%%%r!cH| tj|jz Sr$rr]s r"rzLinearMixing.rmatvecrr!ctjtj|jdd|jz S)Nr)r%rr{r5rrs r"rzLinearMixing.todenses,wwrwwtzz!}bm<==r!cyr$r r s r"r zLinearMixing._updaterr!r$r) rrrrDrrrrrrr r r!r"rrs*,&&> r!rcFeZdZdZd dZdZd dZdZd dZdZ d Z d Z y) ra Find a root of a function, using a tuned diagonal Jacobian approximation. The Jacobian matrix is diagonal and is tuned on each iteration. .. warning:: This algorithm may be useful for specific problems, but whether it will work may depend strongly on the problem. See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='excitingmixing'`` in particular. Parameters ---------- %(params_basic)s alpha : float, optional Initial Jacobian approximation is (-1/alpha). alphamax : float, optional The entries of the diagonal Jacobian are kept in the range ``[alpha, alphamax]``. %(params_extra)s NcXtj|||_||_d|_yr$)rrralphamaxbeta)rrrs r"rzExcitingMixing.__init__s%%    r!ctj||||tj|jdf|j |j |_yrk)rrar%r{r5rr-rrs r"razExcitingMixing.setups=T1a.GGTZZ],djj K r!c"| |jzSr$rr_s r"rzExcitingMixing.solve r$))|r!c"| |jz Sr$rr]s r"rzExcitingMixing.matvec rr!c>| |jjzSr$rrr_s r"rzExcitingMixing.rsolver$)).."""r!c>| |jjz Sr$rr]s r"rzExcitingMixing.rmatvecrr!cFtjd|jz S)Nr)r%rrrs r"rzExcitingMixing.todenseswwr$))|$$r!c ||jzdkD}|j|xx|jz cc<|j|j|<tj|jd|j |jy)Nr)out)rrrr%clipr)rr)rrxr rr incrs r"r zExcitingMixing._updates\}q  $4::%:: 4%  1dmm;r!)NrWrrr r!r"rrs04 L##%>> from scipy.optimize import BroydenFirst, KrylovJacobian >>> from scipy.optimize import InverseJacobian >>> jac = BroydenFirst() >>> kjac = KrylovJacobian(inner_M=InverseJacobian(jac)) If the preconditioner has a method named 'update', it will be called as ``update(x, f)`` after each nonlinear step, with ``x`` giving the current point, and ``f`` the current function value. outer_k : int, optional Size of the subspace kept across LGMRES nonlinear iterations. See `scipy.sparse.linalg.lgmres` for details. inner_kwargs : kwargs Keyword parameters for the "inner" Krylov solver (defined with `method`). Parameter names must start with the `inner_` prefix which will be stripped before passing on the inner method. See, e.g., `scipy.sparse.linalg.gmres` for details. %(params_extra)s See Also -------- root : Interface to root finding algorithms for multivariate functions. See ``method='krylov'`` in particular. scipy.sparse.linalg.gmres scipy.sparse.linalg.lgmres Notes ----- This function implements a Newton-Krylov solver. The basic idea is to compute the inverse of the Jacobian with an iterative Krylov method. These methods require only evaluating the Jacobian-vector products, which are conveniently approximated by a finite difference: .. math:: J v \approx (f(x + \omega*v/|v|) - f(x)) / \omega Due to the use of iterative matrix inverses, these methods can deal with large nonlinear problems. SciPy's `scipy.sparse.linalg` module offers a selection of Krylov solvers to choose from. The default here is `lgmres`, which is a variant of restarted GMRES iteration that reuses some of the information obtained in the previous Newton steps to invert Jacobians in subsequent steps. For a review on Newton-Krylov methods, see for example [1]_, and for the LGMRES sparse inverse method, see [2]_. References ---------- .. [1] C. T. Kelley, Solving Nonlinear Equations with Newton's Method, SIAM, pp.57-83, 2003. :doi:`10.1137/1.9780898718898.ch3` .. [2] D.A. Knoll and D.E. Keyes, J. Comp. Phys. 193, 357 (2004). :doi:`10.1016/j.jcp.2003.08.010` .. [3] A.H. Baker and E.R. Jessup and T. Manteuffel, SIAM J. Matrix Anal. Appl. 26, 962 (2005). :doi:`10.1137/S0895479803422014` Examples -------- The following functions define a system of nonlinear equations >>> def fun(x): ... return [x[0] + 0.5 * x[1] - 1.0, ... 0.5 * (x[1] - x[0]) ** 2] A solution can be obtained as follows. >>> from scipy import optimize >>> sol = optimize.newton_krylov(fun, [0, 0]) >>> sol array([0.66731771, 0.66536458]) Nc J||_||_ttjj j tjj jtjj jtjj jtjj jtjj jj|||_ t||j|_|jtjj jur<||jd<d|jd<|jjddnR|jtjj j tjj j tjj jfvr|jjddn|jtjj jur||jd<d|jd<|jjd g|jjd d |jjd d |jjdd|j#D]6\}}|j%dst'd|z||j|dd<8y)N)bicgstabgmreslgmrescgsminrestfqmr)rqrhrTrrqatolrouter_kouter_vprepend_outer_vTstore_outer_AvFinner_zUnknown parameter %s)preconditionerrrrrrrrrrrrgetmethod method_kw setdefaultgcrotmkr startswithrd) rrr inner_maxiterinner_Mrrkeyrs r"rzKrylovJacobian.__init__s9% \\((11,,%%++<<&&-- ##''<<&&--,,%%++ c&&! mt7J7JK ;;%,,--33 3(5DNN9 %()DNN9 % NN % %fa 0 [[U\\0088"\\0099"\\004466 NN % %fa 0 [[ELL//66 6(/DNN9 %()DNN9 % NN % %i 4 NN % %&7 > NN % %&6 > NN % %fa 0((* ,JC>>(+ !7#!=>>&+DNN3qr7 # ,r!ct|jj}t|jj}|jtd|ztd|z |_y)Nr)rr7r'rromega)rmxmfs r"_update_diff_stepz KrylovJacobian._update_diff_stepsO \    \   ZZ#a*,s1bz9 r!cft|}|dk(rd|zS|j|z }|j|j||zz|jz |z }t j t j|s3t j t j|r td|S)Nrz$Function returned non-finite results) rrrRr7rr%r<r;rd)rr?nvscr[s r"rzKrylovJacobian.matvecs !W 7Q3J ZZ"_ YYtwwA~ & 0B 6vvbkk!n%"&&Q*@CD Dr!cd|jvr-|j|j|fi|j\}}|S|j|j|fd|i|j\}}|S)NrV)rrop)rrhsrVsolrs r"rzKrylovJacobian.solvesg DNN "# DGGSCDNNCIC $ DGGSLcLT^^LIC r!c||_||_|j|j4t |jdr|jj ||yyy)Nri)r7rrrrri)rr)rs r"rizKrylovJacobian.updatesZ      *t**H5##**1a06 +r!ctj||||||_||_tj j j||_|j1tj|jjdz|_ |j|j5t!|jdr|jj|||yyy)Nrra)rrar7rrrraslinearoperatorrrr%rr-rrrr)rr)rrRs r"razKrylovJacobian.setupstQ4(,,%%66t< :: !''*..48DJ      *t**G4##))!Q55 +r!)NrN r) rrrrDrrrrrirar r!r"rr#s2cJCE')-,^: 16r!rc Lt|j}|\}}}}}}} tt|t | d|} dj | D cgc] \} } | d| c} } } | rd| z} dj | D cgc] \} } | d| c} } }|r|dz}|rt d|zd}|t|| |j|z}i}|jtt||||}|j|_ t||Scc} } wcc} } w)a Construct a solver wrapper with given name and Jacobian approx. It inspects the keyword arguments of ``jac.__init__``, and allows to use the same arguments in the wrapper function, in addition to the keyword arguments of `nonlin_solve` Nz, =zUnexpected signature %sa def %(name)s(F, xin, iter=None %(kw)s, verbose=False, maxiter=None, f_tol=None, f_rtol=None, x_tol=None, x_rtol=None, tol_norm=None, line_search='armijo', callback=None, **kw): jac = %(jac)s(%(kwkw)s **kw) return nonlin_solve(F, xin, jac, iter, verbose, maxiter, f_tol, f_rtol, x_tol, x_rtol, tol_norm, line_search, callback) )rrjackwkw)_getfullargspecrlistrr joinrdrrriglobalsexecrDrG)rr signatureargsvarargsvarkwdefaults kwonlyargs kwdefaults_kwargsrIr?kw_strkwkw_strwrappernsrRs r"_nonlin_wrapperrs,  -I@I=D'5(J A #dCM>?+X6 7F YY8A1#Qqe 8 9F yy8AQCq*89Hd?2Y>??G$6s||"*,,G BIIgi" d8D;;DL TN K;99s D D rrrrrrr) rNFNNNNNNrTNFT)rTg:0yE>r):rjnumpyr% scipy.linalgrrrrrrr r r scipy.sparse.linalgr scipy.sparser rscipy._lib._utilr r _linesearchrr__all__ Exceptionrr*r1r9r@rstriprErGrrhr^rrr`rrrrrrrrrrrrrrrrrr r!r"rs ??$$'FC 9 I    (P a1 h/ ?DKO?C48P"f FJ!*Z9<9<@AAH##&Y?@X4AAH * + 0o>od9L9@U~UxA1.A1H+ >+ \8<^8<~C6XC6T)X :| 4 :} 5 :x 0~|< m[9  !1>B@ r!