Q. Speed Up - metrics for performance evaluation?

Speed up is ratio of time needed to execute a given program employing a specific algorithm on a machine with single processor (It implies that T (1) (where n=1)) to time needed to execute same program using a particular algorithm on a machine with multiple processors (It implies that T (n)). Fundamentally speed up factor helps us in knowing relative gain achieved in shifting from a sequential machine to a parallel computer. It can be noted the term T(1) denotes amount of time taken to execute a program employing the best sequential algorithm It implies that the algorithm with least time complexity. 

Let's take an illustration and explain practical use of speedup. Suppose we have a problem of multiplying n numbers. Time complexity of sequential algorithm for a machine with single processor is O(n) because we require one loop for reading and computing output. But in the parallel computer let every number is assigned to specific processor and computation model being used being a hypercube. In such a condition the total number of steps essential to compute result is log n it implies that time complexity is O(log n). Figure below explains the steps to be followed for achieving the required output in a parallel hypercube computer model.

As number of steps followed is equivalent to 3 it implies that log 8 where n is number of processors. Therefore complexity is O(log n). 

In view of fact sequential algorithm takes 8 steps whereas above mentioned parallel algorithm takes 3 steps speed up is as under:

Because value of S(n) is inversely proportional to time needed to calculate output on a parallel number that in turn is dependent on number of processors used for performing the computation. The relation amid S(n) versus Number of processors is displayed in Figure. The speedup is directly proportional to number of processors so a linear arc is portrayed. But in situations where there is parallel overhead the arc is sub-linear.

Figure: Steps followed for multiplying n numbers stored on n processors

Figure: Speed-up vs. Number of Processors