i UddlZddlmcmZddlmZddlmZgZe e e d<ejjGddZy)N)Tensor)2_scripted_functional_optimizer_deprecation_warning__all__ceZdZ ddeedededededededed ed efd Zd ed edzfdZdeedzfdZ y)_FunctionalSGDparamslrmomentum dampening weight_decaynesterovmaximizeforeachfused_allow_empty_param_listc btd||||d|_||_||_||_| |_t jjtt jttt jffi|_ t|dk(r | s tdd|i|_y)N) stacklevel)r r r r rz%optimizer got an empty parameter listr)rdefaultsr rrrtorchjitannotatedictrstrstatelen ValueError param_group) selfrr r r r r rrrrs `/var/www/html/engine/venv/lib/python3.12/site-packages/torch/distributed/optim/functional_sgd.py__init__z_FunctionalSGD.__init__s ;aH "(   !     YY''U\\4U\\@Q;R-R(SUWX v;! $;DE E%f-paramgradNc|jd}|jd}|jd}|jd}|g}g}g} d} |u| j||jrd} ||jvri|j|<|j|} d| vr|jdn|j| dt j 5t j|| ||||||j|j| |j|jdd ddd|j|} |d } | | | d<yy#1swY&xYw) z[Similar to self.step, but operates on a single parameter and its gradient. r r r r FNTmomentum_buffer r r r r r rhas_sparse_gradrr grad_scale found_infr) rappend is_sparserrno_gradFsgdr rrr) rr#r$r r r r rmomentum_buffer_listgradsr(rr&s r step_paramz_FunctionalSGD.step_param:sW }}^4 ==,MM+. ]]4 46   LL ~~"&DJJ&$& 5!JJu%E -$++D1$++E2C,DE ]]_  EE$)!# / jj  $ 5!.q1  &'6E# $ ')  s A EE  gradientsc|jd}g}g}g}|jd}|jd}|jd}|jd} t|t|k7r*tddt|dzd t|zd } t ||D]\} } |  |j | |j | | j rd } | |jvri|j| <|j| } d | vr|j d{|j | d tj5tj||||||| |j|j| |j|jdd dddt!|D]!\}}|j|} ||}||| d <#y#1swY9xYw)Nrr r r r zEthe gradients passed in does not equal to the size of the parameters!zParams length: z. zGradients length: FTr&r')rrrrzipr+r,rrr-r.r/r rrr enumerate)rr3rparams_with_gradr1r0r r r r r(r#gradientripr&s r stepz_FunctionalSGD.stepls!!(+46 ]]4 }}^4 ==,MM+. v;#i. (W#CK=34&s9~&678   "695 JOE8# ''. X&%%&*O *(*DJJu% 5)$E1(//5(//6G0HI J ]]_  EE $)!# / jj  &./ ;DAqJJqME215O*+:'(  ;'  s A G  G) g{Gz?r<r<FFFFF) __name__ __module__ __qualname__listrfloatboolr!r2r;r"r rrs !(-.V . . .  .  ....."&.B0707ftm07d9;d6D=19;r"r)rtorch.optim._functionaloptim _functionalr.r,torch.distributed.optim._deprecation_warningrrr@r__annotations__rscriptrrCr"r rJsN ## cM;M;M;r"