Home   >   CSC-OpenAccess Library   >    Manuscript Information
Analysis & Integrated Modeling of the Performance Evaluation Techniques for Evaluating Parallel Systems.
Amit Chhabra, Gurvinder Singh
Pages - 1 - 10     |    Revised - 15-06-2007     |    Published - 30-06-2007
Volume - 1   Issue - 1    |    Publication Date - June 2007  Table of Contents
MORE INFORMATION
KEYWORDS
Integrated model, Metrics, Parallel systems, Performance, Evaluation
ABSTRACT
Parallel computing has emerged as an environment for computing inherently parallel and computation intensive applications. Performance is always a key factor in determining the success of any system. So parallel computing systems are no exception. Evaluating and analyzing the performance of parallel systems is an important aspect of parallel computing research. Evaluating and analyzing parallel system is difficult due to the complex interaction between application characteristics and architectural features. Experimental measurement, Theoretical/Analytical modeling and Simulation are the most widely used techniques in the performance evaluation of parallel systems. Experimental measurement uses real or synthetic workloads, usually known as benchmarks, to evaluate and analyze their performance on actual hardware. Theoretical/Analytical models try to abstract details of a parallel system. Simulation and other performance monitoring/visualization tools are extremely popular because they can capture the dynamic nature of the interaction between applications and architectures. Each of them has several types. For example, Experimental measurement has software, hardware, and hybrid. Theoretical/Analytical modeling has queueing network, Petri net, etc. and simulation has discrete event, trace/execution driven, Monte Carlo. Each of these three techniques has their own pros and cons. The purpose of this paper is firstly to present a qualitative parametric comparative analysis of these techniques based on parameters like stage, output statistics, accuracy, cost, resource consumption, time consumption, flexibility, scalability, tools required, trustability and secondly to justify the need for an integrated model combining the advantages of all these techniques to evaluate the performance of parallel systems and thirdly to present a new integrated model for performance evaluation . This paper also discusses certain issues like selecting an appropriate metric for evaluating parallel systems.
CITED BY (3)  
1 Fritzsche, M., & Gilani, W. (2013). U.S. Patent No. 8,438,533. Washington, DC: U.S. Patent and Trademark Office.
2 Elkabbany, G. F., & Elnemr, H. A. (2013). An Efficient Parallel Based Diabetic Retinopathy Grading Algorithm. International Journal of Computer Science and Information Security, 11(9), 12.
3 Younes, O., & Thomas, N. (2013). Modelling and performance analysis of multi-hop ad hoc networks. Simulation Modelling Practice and Theory, 38, 69-97.
1 Google Scholar 
2 Academic Journals Database 
3 ScientificCommons 
4 Academic Index 
5 CiteSeerX 
6 iSEEK 
7 Socol@r  
8 ResearchGATE 
9 Libsearch 
10 Bielefeld Academic Search Engine (BASE) 
11 Scribd 
12 WorldCat 
13 SlideShare 
14 PDFCAST 
15 PdfSR 
16 Google Books 
17 Chinese Directory Of Open Access 
12]Jelly, I. ,Gorton, I., “Software engineering for parallel systems”, Information and Software Technology, vol. 36, no. 7, pp. 381-396, 1994.
12]Jelly, I. ,Gorton, I., Software engineering for parallel systems, Information and Software Technology, vol. 36, no. 7, pp. 381-396, 1994.
28]L. G. Valiant. “A Bridging Model for Parallel Computation” Communications of the ACM, 33(8):103–111, August 1990.
28]L. G. Valiant. A Bridging Model for Parallel Computation Communications of the ACM, 33(8):103111, August 1990.
2]E.Caromona ,M.Rice, “Modelling the serial and parallel fractions of a parallel algorithm”,Journal of Parallel and Distributed Computing,13,286-298,1991.
2]E.Caromona ,M.Rice, Modelling the serial and parallel fractions of a parallel algorithm,Journal of Parallel and Distributed Computing,13,286-298,1991.
A. Aggarwal, A. K. Chandra, and M. Snir “ On Communication Latency in PRAM Computations” In Proceedings of the First Annual ACM Symposium on Parallel Algorithms and Architectures, pages 11–21, 1989.
A. Aggarwal, A. K. Chandra, and M. Snir On Communication Latency in PRAM Computations In Proceedings of the First Annual ACM Symposium on Parallel Algorithms and Architectures, pages 1121, 1989.
Agarwal, A., “Performance tradeoffs in multithreaded processors,” IEEE Transactions on Parallel and distributed Systems, vol. 3, no. 5, pp. 525-539, September 1992.
Agarwal, A., Performance tradeoffs in multithreaded processors, IEEE Transactions on Parallel and distributed Systems, vol. 3, no. 5, pp. 525-539, September 1992.
Covington, R. G., Dwarkadas, S., Jump, J. R., Sinclair, J. B. ,Madala, S., “The efficient simulation of parallel computer systems,” International Journal in Computer Simulation, vol. 1, pp.31-58, 1991.
Covington, R. G., Dwarkadas, S., Jump, J. R., Sinclair, J. B. ,Madala, S., The efficient simulation of parallel computer systems, International Journal in Computer Simulation, vol. 1, pp.31-58, 1991.
D. Bailey et al. The NAS Parallel Benchmarks. International Journal of Supercomputer Applications, 5(3):63–73, 1991.
D. Bailey et al. The NAS Parallel Benchmarks. International Journal of Supercomputer Applications, 5(3):6373, 1991.
D. Culler et al.”LogP: Towards a realistic model of parallel computation”In Proceedings of the 4th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pages 1–12, May 1993.
D. Culler et al.LogP: Towards a realistic model of parallel computationIn Proceedings of the 4th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pages 112, May 1993.
D. F. Vrsalovic, D. P. Siewiorek, Z. Z. Segall, and E. Gehringer “Performance Prediction and Calibration for a Class of Multiprocessors” IEEE Transactions on Computer Systems, 37(11):1353–1365, November 1988.
D. F. Vrsalovic, D. P. Siewiorek, Z. Z. Segall, and E. Gehringer Performance Prediction and Calibration for a Class of Multiprocessors IEEE Transactions on Computer Systems, 37(11):13531365, November 1988.
D.Nussbam and A.Agrawal, “ Scalability of parallel machines”,CACM,34,3,57-61,1991.
D.Nussbam and A.Agrawal, Scalability of parallel machines,CACM,34,3,57-61,1991.
Ferrari, D., “Considerations on the insularity of performance evaluation”, Performance Evaluation Review, vol. 14, no. 2, pp. 21-32, August 1986.
Ferrari, D., Considerations on the insularity of performance evaluation, Performance Evaluation Review, vol. 14, no. 2, pp. 21-32, August 1986.
H. Alt, T. Hagerup, K. Mehlhorn, F. P. Preparata “ Deterministic Simulation of Idealized Parallel Computers on More Realistic Ones” SIAM Journal of Computing, 16(5):808–835, 1987.
H. Alt, T. Hagerup, K. Mehlhorn, F. P. Preparata Deterministic Simulation of Idealized Parallel Computers on More Realistic Ones SIAM Journal of Computing, 16(5):808835, 1987.
Hadsell, R. W., Keinzle, M. G. and Milliken, K. R., “The hybrid monitor system,” Technical Report RC9339, IBM Thomas J. Watson Research Center, New York, 1983.
Hadsell, R. W., Keinzle, M. G. and Milliken, K. R., The hybrid monitor system, Technical Report RC9339, IBM Thomas J. Watson Research Center, New York, 1983.
Hughes, J. H., “Diamond ¾ A digital analyzer and monitoring device,” Performance Evaluation Review, vol. 9, no. 2, pp. 27-34, 1980.
Hughes, J. H., Diamond A digital analyzer and monitoring device, Performance Evaluation Review, vol. 9, no. 2, pp. 27-34, 1980.
Ibbett, R., “The hardware monitoring of a high performance processor,” in: Benwell, N. (ed), Computer Performance Evaluation, Cranfield Institute of Technology, UK, pp. 274-292, December 1978.
Ibbett, R., The hardware monitoring of a high performance processor, in: Benwell, N. (ed), Computer Performance Evaluation, Cranfield Institute of Technology, UK, pp. 274-292, December 1978.
J. P. Singh, W-D. Weber, and A. Gupta. SPLASH: Stanford Parallel Applications for Shared-Memory. Technical Report CSL-TR-91-469, Computer Systems Laboratory, Stanford University, 1991.
J. P. Singh, W-D. Weber, and A. Gupta. SPLASH: Stanford Parallel Applications for Shared-Memory. Technical Report CSL-TR-91-469, Computer Systems Laboratory, Stanford University, 1991.
J.Gustafson, “Reevaluating Amdahl’s Law”, CACM, 31,5,532-533,1988.
J.Gustafson, Reevaluating Amdahls Law, CACM, 31,5,532-533,1988.
J.JaJa, “ An introduction to parallel algorithms”,Addison Wesley,1992.
J.JaJa, An introduction to parallel algorithms,Addison Wesley,1992.
J.Worlton,“Toward a taxonomy of performance metrics”, Parallel Computing 17(10-11): 1073-1092 (1991) .
J.Worlton,Toward a taxonomy of performance metrics, Parallel Computing 17(10-11): 1073-1092 (1991) .
Jain, R., The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling, John Wiley & Sons, New York, 1991.
Jain, R., The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling, John Wiley & Sons, New York, 1991.
M.Berryetal. The Perfect Club Benchmarks: Effective Performance Evaluation of Supercomputers. International Journal of Supercomputer Applications, 3(3):5–40, 1989.
M.Berryetal. The Perfect Club Benchmarks: Effective Performance Evaluation of Supercomputers. International Journal of Supercomputer Applications, 3(3):540, 1989.
Malony, A. D., Reed, D. A. ,Wijshoff, H. A. G., “Performance measurement intrusion and perturbation analysis,” IEEE Transactions on Parallel and Distrubuted Systems, vol. 3, no. 4, pp. 443-450, July 1992.
Malony, A. D., Reed, D. A. ,Wijshoff, H. A. G., Performance measurement intrusion and perturbation analysis, IEEE Transactions on Parallel and Distrubuted Systems, vol. 3, no. 4, pp. 443-450, July 1992.
Menasce, D. A. ,Barroso, L. A., “A methodology for performance evaluation of parallel applications on multiprocessors,” Journal of Parallel and Distributed Computing, vol. 14, pp. 1-14,1992.
Menasce, D. A. ,Barroso, L. A., A methodology for performance evaluation of parallel applications on multiprocessors, Journal of Parallel and Distributed Computing, vol. 14, pp. 1-14,1992.
P. B. Gibbons. A More Practical PRAM Model. In Proceedings of the First Annual ACM Symposium on Parallel Algorithms and Architectures, pages 158–168, 1989.
P. B. Gibbons. A More Practical PRAM Model. In Proceedings of the First Annual ACM Symposium on Parallel Algorithms and Architectures, pages 158168, 1989.
Plattner, B. ,Nievergelt, J., “Monitoring program execution: A survey,” IEEE Computer, vol. 14, pp. 76- 93, November 1981.
Plattner, B. ,Nievergelt, J., Monitoring program execution: A survey, IEEE Computer, vol. 14, pp. 76- 93, November 1981.
Power, L. R., “Design and use of a program execution analyzer,” IBM Systems Journal, vol. 22,no. 3, pp. 271-294, 1983.
Power, L. R., Design and use of a program execution analyzer, IBM Systems Journal, vol. 22,no. 3, pp. 271-294, 1983.
Ries, B., Anderson, R., Auld, W., Breazeal, D., Callaghan, K., Richards, E. and Smith, W., “The Paragon performance monitoring environment,” Proceedings of the conference on Supercomputing’93, pp. 850-859, 1993.
Ries, B., Anderson, R., Auld, W., Breazeal, D., Callaghan, K., Richards, E. and Smith, W., The Paragon performance monitoring environment, Proceedings of the conference on Supercomputing93, pp. 850-859, 1993.
S. Fortune and J. Wyllie. Parallelism in random access machines. In Proceedings of the 10th Annual Symposium on Theory of Computing, pages 114–118, 1978.
S. Fortune and J. Wyllie. Parallelism in random access machines. In Proceedings of the 10th Annual Symposium on Theory of Computing, pages 114118, 1978.
S.Ranka, S.Sahni, “Hypercube algorithms”,Springer-Verlag,New York,1990.
S.Ranka, S.Sahni, Hypercube algorithms,Springer-Verlag,New York,1990.
V.Kumar,V.Nageshwara and K.Ramesh, “Parallel depth first search on the ring architecture”, Proc.1988 International Conference on Parallel Processing,Penn. State Univ. Press,128-132,1988.
V.Kumar,V.Nageshwara and K.Ramesh, Parallel depth first search on the ring architecture, Proc.1988 International Conference on Parallel Processing,Penn. State Univ. Press,128-132,1988.
X.Sun ,D.Rover, “Scalability of parallel algorithm-machine combinations”,IEEE Transactions Of Parallel and Distributed systems,5,6,599-613,1994.
X.Sun ,D.Rover, Scalability of parallel algorithm-machine combinations,IEEE Transactions Of Parallel and Distributed systems,5,6,599-613,1994.
X.Sun ,J.Gustafson, “Towards a better parallel performance metric”,Parallel Computing,17,1093- 1109,1991.
X.Sun ,J.Gustafson, Towards a better parallel performance metric,Parallel Computing,17,1093- 1109,1991.
X.Sun ,L.Ni, “ Scalable problems and memory-bouneded speedup”, Journal of Parallel and Distributed Computing,19,27-37,1993.
X.Sun ,L.Ni, Scalable problems and memory-bouneded speedup, Journal of Parallel and Distributed Computing,19,27-37,1993.
X.Sun, L.Ni, “Another view on parallel speedup”,Proceedings Supercomputing 90,324-333,1990.
X.Sun, L.Ni, Another view on parallel speedup,Proceedings Supercomputing 90,324-333,1990.
Dr. Amit Chhabra
- India
chhabra_amit78@yahoo.com
Mr. Gurvinder Singh
- India