Fault-Tolerant Skeleton Program Execution in Distributed Heterogeneous Parallel Systems

Sammanfattning

Distributed stream-processing pipelines deployed on heterogeneous systems can stop producing useful output when a worker node fails. The same problem can occur when one pipeline task is affected by a partial node failure while the node remains reachable to the orchestrator. SkePU-Streaming currently leaves such failures to the surrounding platform or application code. That leaves a gap for long-running pipelines that need the runtime itself to detect a failure, remap the affected pipeline tasks, and reconnect the stream. The prototype adds framework-level recovery support to SkePU-Streaming. It coor- dinates recovery inside the runtime and restores the stream path after a failure trigger. The evaluation uses controlled fault injection and repeated timing measurements on distributed pipelines. For recovery scope, the evaluation tests six fault cases that expose the core one-to-one, one-to-many, and many-to-one local recovery patterns, and checks the same patterns inside a larger motion-detection pipeline. In these cases, the prototype detects the injected failures, remaps the affected tasks, restores the expected stream path, and passes the recovery checks. For the evaluated producer–consumer workload, enabling recovery support leaves the mean per-run median latency in the same range (0.97 ms vs. 1.09 ms), with no latency penalty distinguishable from run-to-run variation. Warm standby reduces the mean post-detection recovery interval from 24.03 s under cold failover to 54.69 ms. The results show that failure detection, task remapping, and bounded recovery sup- port can be integrated into SkePU-Streaming. Within the evaluated cases, skeleton ap- plications resume streaming after the runtime detects a failure and repairs the affected path.