Q. What is Single Program Multiple Data?

A general style of writing data parallel programs for MIMD computers is SPMD (single program, multiple data) means all processors execute same program however every processor operates on a separate part of problem data. It's easier to program than true MIMD however more flexible than SIMD. Though most parallel computers today are MIMD architecturally they are generally programmed in SPMD style. In this style though there is not any central controller the worker nodes carry on doing basically the same thing at effectively same time. Rather than central copies of control variables stored on control processor of a SIMD computer, control variables (iteration counts and so on) are generally stored in a replicated fashion across MIMD nodes. Every node has its own local copy of these global control variables however each node updates them in an identical way. There are no centrally issued parallel instructions however communications generally occur in well-defined collective phases. These data exchanges happen in a prefixed manner which implicitly or explicitly synchronizes the peer nodes. The condition is something such as an orchestra without a conductor. There is no central control however every particular plays from the same script. The group as a complete stays in lockstep. This loosely synchronous style has a number of similarities to Bulk Synchronous Parallel (BSP) model of computing introduced by theorist Les Valiant in early 1990's. The limited pattern of the collective synchronization is easier to deal with than complex synchronisation problems of a general concurrent programming.