Selecting the Research Method

As a student, it is important to choose an appropriate research method that suits solving the problem at hand. Please look at the links below (most are accessible from within the university network) and discuss with your supervisor which is the best approach. Note that you do not need to write lengthy chapters about the method, it is just necessary to clearly state what the method you use is and how your approach is adjusted to the appropriate method. Please also note that the structure of a thesis is very similar to that of a research paper.

How to Write and Read a Scientific Evaluation Paper

by Roel Wieringa, Hans Heerkens and Björn Regnell.

"Scientific evaluation papers investigate existing problem situations or validate proposed solutions with scientific means, such as by experiment or case study. There is a growing amount of literature about how to report about empirical research in software engineering, but there is still some confusion about the difference between a scientific evaluation paper and other kinds of research papers. This is related to lack of clarity about the relation between empirical research, engineering, and industrial practice. In this minitutorial we give a brief rundown on how to structure a scientific evaluation papers as a special kind of research paper, using experiment reports and case study reports as examples. We give checklists of items that a reader should be able to find in these papers, and sketch the dilemmas that writers and readers of these papers face when applying these checklists"

A design science research methodology and its application to accounting information systems research

by Guido L.Geerts

"Natural science research follows a stereotypical pattern and such uniformity makes it easier to recognize and evaluate the results of such research. A similar format has been lacking for design science research. This issue was addressed by Peffers et al. (2008) who defined such a template for design science research for information systems: the design science research methodology (DSRM). In this paper, we first discuss design science research and the DSRM. Then, we illustrate the application of the DSRM to AIS research through retroactive analysis. Finally, we integrate the DSRM into the operational specification of artifact networks and use the REA literature for illustration purposes"

The above paper gives a nice overview of how design science research method is applied to concrete information systems. This may be easier to read than the classic references Design Science Research in Information Systems by A. Hevner and S. Chatterjee, and A Design Science Research Methodology for Information Systems Research by Ken Peffers, Tuure Tuunanen, Marcus A. Rothenberger and Samir Chatterjee.

Selecting Empirical Methods for Software Engineering Research

by Steve Easterbrook, Janice Singer, Margaret-Anne Storey and Daniela Damian.

"Selecting a research method for empirical software engineering research is problematic because the benefits and challenges to using each method are not yet well catalogued. Therefore, this chapter describes a number of empirical methods available. It examines the goals of each and analyzes the types of questions each best addresses. Theoretical stances behind the methods, practical considerations in the application of the methods and data collection are also briefly reviewed. Taken together, this information provides a suitable basis for both understanding and selecting from the variety of methods applicable to empirical software engineering."

ACM SIGPLAN Empirical Evaluation Guidelines

The programming languages research community often develops ideas whose worth is evaluated empirically. Compiler optimizations, static and dynamic analyses, program synthesizers, testing tools, memory management algorithms, new language features, and other research developments each depend on some empirical evidence to demonstrate their effectiveness. This reality raises some important questions. What kind of empirical evidence yields the most reliable conclusions? What are the best practices for putting together an empirical evaluation in PL research? Do PL research papers published in top venues always follow these best practices?

To answer these questions, in August of 2017 the SIGPLAN Executive Committee formed the ad hoc committee on Programming Language Research Empirical Evaluations. The committee is chaired by Steve Blackburn, and its members include Matthias Hauswirth, Emery Berger, and Michael Hicks. Shriram Krishnamurthi has acted as an external collaborator. The committee brings together expertise on empirical evaluation methodology, experience in running workshops and publishing papers on that topic, experience introducing artifact evaluation into SIGPLAN conferences, and experience chairing the PCs of major SIGPLAN conferences.

Statistical Methods

Meaningfully designed statistical analysis is at the core of many theses. Please discuss the choice of appropriate methods with your supervisor. A very nice overview of how statistical methods have evolved during the computer age is given in Computer Age Statistical Inference by Bradley Efron and Trevor Hastie.

R3: repeatability, reproducibility and rigor

by Jan Vitek and Tomas Kalibera.

"Computer systems research spans subdisciplines that include embedded systems, programming languages and compilers, networking, and operating systems. Our contention is that a number of structural factors inhibit quality systems research. We highlight some of the factors we have encountered in our own work and observed in published papers and propose solutions that could both increase the productivity of researchers and the quality of their output."

Case Study Research in Software Engineering: Guidelines and Examples

by Per Runeson, Martin Höst, Austen Rainer and Björn Regnell.

"Based on their own experiences of in-depth case studies of software projects in international corporations, in this book the authors present detailed practical guidelines on the preparation, conduct, design and reporting of case studies of software engineering. This is the first software engineering specific book on the case study research method."

The book is accessible via Ebook Central. Please go to e-resource portal Primo. Then enter the title in quotation marks. There will be 2 responses both linking to the EbookCentral database. From outside the university network please follow the instructions at the web page of the IT Department.

Raamat on TTÜ tudengitele ja töötajatele kättesaadav andmebaasi Ebook Central kaudu. Kõige kiiremini jõuab raamatuni raamatukogu esilehel Infoallikad loetelus e-ressursside portaali Primo kasutades. Kui sisestada raamatu pealkiri jutumärkides, siis saame kaks vastust, mis mõlemad viivad EbookCentral andmebaasi. Väljaspool ülikooli võrku saavad üliõpilased juurdepääsu ülikooli Uni-ID-ga toimides vastavalt IT Helpdeski juhendile.

Research Methods in Education

by Louis Cohen, Lawrence Manion and Keith Morrison.

If your thesis touches on improving or analysis of the way education is provided, please consult the book linked above. Please note that for IT students the thesis should involve a substantial amount of software development or data analysis, but you should be aware of the methods used for education research and be able to fit your results into the wider context.