Home   >   CSC-OpenAccess Library   >    Manuscript Information
Full Text Available

This is an Open Access publication published under CSC-OpenAccess Policy.
Analysis & Integrated Modeling of the Performance Evaluation Techniques for Evaluating Parallel Systems.
Amit Chhabra, Gurvinder Singh
Pages - 1 - 10     |    Revised - 15-06-2007     |    Published - 30-06-2007
Volume - 1   Issue - 1    |    Publication Date - June 2007  Table of Contents
Integrated model, Metrics, Parallel systems, Performance, Evaluation
Parallel computing has emerged as an environment for computing inherently parallel and computation intensive applications. Performance is always a key factor in determining the success of any system. So parallel computing systems are no exception. Evaluating and analyzing the performance of parallel systems is an important aspect of parallel computing research. Evaluating and analyzing parallel system is difficult due to the complex interaction between application characteristics and architectural features. Experimental measurement, Theoretical/Analytical modeling and Simulation are the most widely used techniques in the performance evaluation of parallel systems. Experimental measurement uses real or synthetic workloads, usually known as benchmarks, to evaluate and analyze their performance on actual hardware. Theoretical/Analytical models try to abstract details of a parallel system. Simulation and other performance monitoring/visualization tools are extremely popular because they can capture the dynamic nature of the interaction between applications and architectures. Each of them has several types. For example, Experimental measurement has software, hardware, and hybrid. Theoretical/Analytical modeling has queueing network, Petri net, etc. and simulation has discrete event, trace/execution driven, Monte Carlo. Each of these three techniques has their own pros and cons. The purpose of this paper is firstly to present a qualitative parametric comparative analysis of these techniques based on parameters like stage, output statistics, accuracy, cost, resource consumption, time consumption, flexibility, scalability, tools required, trustability and secondly to justify the need for an integrated model combining the advantages of all these techniques to evaluate the performance of parallel systems and thirdly to present a new integrated model for performance evaluation . This paper also discusses certain issues like selecting an appropriate metric for evaluating parallel systems.
CITED BY (3)  
1 Fritzsche, M., & Gilani, W. (2013). U.S. Patent No. 8,438,533. Washington, DC: U.S. Patent and Trademark Office.
2 Elkabbany, G. F., & Elnemr, H. A. (2013). An Efficient Parallel Based Diabetic Retinopathy Grading Algorithm. International Journal of Computer Science and Information Security, 11(9), 12.
3 Younes, O., & Thomas, N. (2013). Modelling and performance analysis of multi-hop ad hoc networks. Simulation Modelling Practice and Theory, 38, 69-97.
1 Google Scholar
2 Academic Journals Database
3 ScientificCommons
4 Academic Index
5 CiteSeerX
7 Socol@r
8 ResearchGATE
9 Libsearch
10 Bielefeld Academic Search Engine (BASE)
11 Scribd
12 WorldCat
13 SlideShare
15 PdfSR
16 Google Books
17 Chinese Directory Of Open Access
1 J.Gustafson, “Reevaluating Amdahl’s Law”, CACM, 31,5,532-533,1988.
2 2]E.Caromona ,M.Rice, “Modelling the serial and parallel fractions of a parallel algorithm”,Journal of Parallel and Distributed Computing,13,286-298,1991.
3 J.JaJa, “ An introduction to parallel algorithms”,Addison Wesley,1992.
4 D.Nussbam and A.Agrawal, “ Scalability of parallel machines”,CACM,34,3,57-61,1991.
5 S.Ranka, S.Sahni, “Hypercube algorithms”,Springer-Verlag,New York,1990.
6 X.Sun, L.Ni, “Another view on parallel speedup”,Proceedings Supercomputing 90,324-333,1990.
7 X.Sun ,J.Gustafson, “Towards a better parallel performance metric”,Parallel Computing,17,1093- 1109,1991.
8 X.Sun ,L.Ni, “ Scalable problems and memory-bouneded speedup”, Journal of Parallel and Distributed Computing,19,27-37,1993.
9 X.Sun ,D.Rover, “Scalability of parallel algorithm-machine combinations”,IEEE Transactions Of Parallel and Distributed systems,5,6,599-613,1994.
10 V.Kumar,V.Nageshwara and K.Ramesh, “Parallel depth first search on the ring architecture”, Proc.1988 International Conference on Parallel Processing,Penn. State Univ. Press,128-132,1988.
11 J.Worlton,“Toward a taxonomy of performance metrics”, Parallel Computing 17(10-11): 1073-1092 (1991) .
12 12]Jelly, I. ,Gorton, I., “Software engineering for parallel systems”, Information and Software Technology, vol. 36, no. 7, pp. 381-396, 1994.
13 Ferrari, D., “Considerations on the insularity of performance evaluation”, Performance Evaluation Review, vol. 14, no. 2, pp. 21-32, August 1986.
14 Plattner, B. ,Nievergelt, J., “Monitoring program execution: A survey,” IEEE Computer, vol. 14, pp. 76- 93, November 1981.
15 Power, L. R., “Design and use of a program execution analyzer,” IBM Systems Journal, vol. 22,no. 3, pp. 271-294, 1983.
16 Malony, A. D., Reed, D. A. ,Wijshoff, H. A. G., “Performance measurement intrusion and perturbation analysis,” IEEE Transactions on Parallel and Distrubuted Systems, vol. 3, no. 4, pp. 443-450, July 1992.
17 Ibbett, R., “The hardware monitoring of a high performance processor,” in: Benwell, N. (ed), Computer Performance Evaluation, Cranfield Institute of Technology, UK, pp. 274-292, December 1978.
18 Ries, B., Anderson, R., Auld, W., Breazeal, D., Callaghan, K., Richards, E. and Smith, W., “The Paragon performance monitoring environment,” Proceedings of the conference on Supercomputing’93, pp. 850-859, 1993.
19 Hadsell, R. W., Keinzle, M. G. and Milliken, K. R., “The hybrid monitor system,” Technical Report RC9339, IBM Thomas J. Watson Research Center, New York, 1983.
20 Hughes, J. H., “Diamond ¾ A digital analyzer and monitoring device,” Performance Evaluation Review, vol. 9, no. 2, pp. 27-34, 1980.
21 Jain, R., The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling, John Wiley & Sons, New York, 1991.
22 M.Berryetal. The Perfect Club Benchmarks: Effective Performance Evaluation of Supercomputers. International Journal of Supercomputer Applications, 3(3):5–40, 1989.
23 D. Bailey et al. The NAS Parallel Benchmarks. International Journal of Supercomputer Applications, 5(3):63–73, 1991.
24 J. P. Singh, W-D. Weber, and A. Gupta. SPLASH: Stanford Parallel Applications for Shared-Memory. Technical Report CSL-TR-91-469, Computer Systems Laboratory, Stanford University, 1991.
25 S. Fortune and J. Wyllie. Parallelism in random access machines. In Proceedings of the 10th Annual Symposium on Theory of Computing, pages 114–118, 1978.
26 P. B. Gibbons. A More Practical PRAM Model. In Proceedings of the First Annual ACM Symposium on Parallel Algorithms and Architectures, pages 158–168, 1989.
27 D. Culler et al.”LogP: Towards a realistic model of parallel computation”In Proceedings of the 4th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pages 1–12, May 1993.
28 28]L. G. Valiant. “A Bridging Model for Parallel Computation” Communications of the ACM, 33(8):103–111, August 1990.
29 A. Aggarwal, A. K. Chandra, and M. Snir “ On Communication Latency in PRAM Computations” In Proceedings of the First Annual ACM Symposium on Parallel Algorithms and Architectures, pages 11–21, 1989.
30 H. Alt, T. Hagerup, K. Mehlhorn, F. P. Preparata “ Deterministic Simulation of Idealized Parallel Computers on More Realistic Ones” SIAM Journal of Computing, 16(5):808–835, 1987.
31 D. F. Vrsalovic, D. P. Siewiorek, Z. Z. Segall, and E. Gehringer “Performance Prediction and Calibration for a Class of Multiprocessors” IEEE Transactions on Computer Systems, 37(11):1353–1365, November 1988.
32 Agarwal, A., “Performance tradeoffs in multithreaded processors,” IEEE Transactions on Parallel and distributed Systems, vol. 3, no. 5, pp. 525-539, September 1992.
33 Menasce, D. A. ,Barroso, L. A., “A methodology for performance evaluation of parallel applications on multiprocessors,” Journal of Parallel and Distributed Computing, vol. 14, pp. 1-14,1992.
34 Covington, R. G., Dwarkadas, S., Jump, J. R., Sinclair, J. B. ,Madala, S., “The efficient simulation of parallel computer systems,” International Journal in Computer Simulation, vol. 1, pp.31-58, 1991.
Dr. Amit Chhabra
- India
Mr. Gurvinder Singh
- India