elastic_execute - vLLM
Skip to content

vllm.distributed.elastic_ep.elastic_execute

Functions:

can_reuse_fused_moe_kernel(parallel_config)

Whether fused MoE kernels can be reused across EP rank changes.

The EP backend must preserve its CUDA graph entries when ranks are connected, disconnected, masked, or unmasked.

Source code in vllm/distributed/elastic_ep/elastic_execute.py 
67
68
69
70
71
72
73
def can_reuse_fused_moe_kernel(parallel_config: ParallelConfig) -> bool:
    """Whether fused MoE kernels can be reused across EP rank changes.

    The EP backend must preserve its CUDA graph entries when ranks are connected,
    disconnected, masked, or unmasked.
    """
    return parallel_config.all2all_backend == "nixl_ep"