iK <ddlZddlmZddlmZmZddlZddlmZddl m Z ddl m Z ddgZ d ejeezeed fzeeefzd ejdzfd Z dd ee deedeeeefdzdeeej*zdzdzd eef dZy)N)Sequence)Anycast)ExceptionWrapper)_get_device_index)Module get_a_varparallel_applyobj.returncpt|tjr|St|ttfr3t t |D] }t|tjs|cSt|trAt t |jD] }t|tjs|cSy)N) isinstancetorchTensorlisttuplemapr dictitems)r results Z/var/www/html/engine/venv/lib/python3.12/site-packages/torch/nn/parallel/parallel_apply.pyr r s#u||$ #e}%)S) F&%,,/  #t)SYY[1 F&%,,/   modulesinputs kwargs_tupdevicesct|t|k(sJdt|dt||t|t|k(s,Jttttfift|z}|t|t|k(sJdgt|z}|Dcgc]}t |d}}|Dcgc]!}t jj|#}}t jjsJdt jjjtjit jt jc ddt dt"dtd tttfd t t j$zdzd t j&dzd dffd }t|dkDrt)t+|||||dD cgc](\}\}} } } } tj,|||| | | | f*} } } } }}} | D]}|j/| D]}|j1n|d|d|d|d|d|dg}t3t|D]8}|}t5|t6r|j9|j;|:|Scc}wcc}wcc} } } } }}w)aApply each `module` in :attr:`modules` in parallel on each of :attr:`devices`. Args: modules (Module): modules to be parallelized inputs (tensor): inputs to the modules devices (list of int or torch.device): CUDA devices :attr:`modules`, :attr:`inputs`, :attr:`kwargs_tup` (if given), and :attr:`devices` (if given) should all have same length. Moreover, each element of :attr:`inputs` can either be a single object as the only argument to a module, or a collection of positional arguments. zThe number of modules z& is not equal to the number of inputs NTzNo available accelerator found.imoduleinputkwargsdevicestreamr cHtj |xYw#1swYyxYw#t$r0 5td|d| |<dddYy#1swYYyxYwwxYw)Nz in replica zQ, no device was provided and no tensor input was found; device cannot be resolved)where)enabledz on device )rset_grad_enabledr r get_devicerr"index acceleratorcurrent_stream device_indexampautocastrr Exception) rrr r!r"r#toutputautocast_enabled device_type grad_enabledlockresultss r_workerzparallel_apply.._workerGs |, >% Ay!1 +A3/44"GAJ \\^F fell +\\F >&&55f=F !!..v6 2 2 "";8H"I 2 "%$7"HE1&1 2 2 2 $#  $ $+  2 2 2 2 2 2 $ $  -'s+fX>     sD,E(<E?"E!"D8E E E(E#E(,D58E =EE EEE(E%!E(%E(( F!3F F!F F!F!)strict)targetargsrNN)lenrrstrrrrr*r+ is_availablecurrent_acceleratortype threadingLockis_grad_enabledis_autocast_enabledintrr"Stream enumeratezipThreadstartjoinrangerrreraiseappend)rrrrxstreamsr7rrr r!r"r#threadsthreadoutputsr1r2r3r4r5r6s @@@@@rr r s$ w<3v; & W.TUXY_U`Tab &7|s:...4S>2.03w<? 7|s7|+++&3w<'3:;a D);G;>K >> DG  !!##L"-1&* ' '''S#X ' ell"T) '  t# ' ''R 7|a ?HGVZ'$O?   ;:FE666   a%O    F LLN  F KKM  71:vay*Q-WQZPG 3v;  f. / NN v  NW<Dh s/K&K# -K(r<)rBcollections.abcrtypingrrr torch._utilsrtorch.cuda._utilsrtorch.nn.modulesr__all__rrrrr r>rFr"r rrr\s$ )/# ( ) S !E#s(O 3d38n D \\D(37:> h f h SMhc3h(4/hcELL(4/ 04 7 h  #Y hr