Example case study research in software engineering

An approach to presenting your own research is to identify an existing academic/scholarly article that reports in some way that you find informative and that might steer your analysis of your own case study or development/maintenance management system.
The following 'convenience sample' of articles ranges over diverse perspectives but offer (in my opinion) useful and informative case studies researching empirical software engineering.

A field study of the software design process for large systems
B Curtis, H Krasner, N Iscoe - Communications of the ACM, 1988 - dl.acm.org

Ethnographically-informed systems design for air traffic control
R Bentley, JA Hughes, D Randall, T Rodden… - Proceedings of the …, 1992 - dl.acm.org

ISO9000 and the very small firm
EM Wareham - IEE Review, 1994 - ieeexplore.ieee.org

Conversational Conventions and Participation in Cross-Functional Design Teams
E Wynn, D G Novick - Proceedings of COOCS, 1995 - ACM

Flight of the Eagle: The Birthing and Life of a Super-Minicomputer.
J Faughnan, S Stevanovic - Project Management, 1996 - faughnan.com

Embracing change with extreme programming
K Beck - Computer, 1999 - ieeexplore.ieee.org

A case study of open source software development: the Apache server
A Mockus, RT Fielding… - Software Engineering, …, 2000 - ieeexplore.ieee.org

Working artefacts: ethnomethods of the prototype
L Suchman, R Trigg, J Blomberg - The British journal of …, 2002 - Wiley Online Library

Reflective Design Practices in Human Computer Interaction and Software EngineeringELC Law - Proceedings of the Workshop on Designing for …, 2004 - Citeseer

When software engineers met research scientists: A case studyJ Segal - Empirical Software Engineering, 2005 - Springer

Sociomaterial practices: Exploring technology at work
WJ Orlikowski - Organization studies, 2007 - oss.sagepub.com

Scrum in a multiproject environment: An ethnographically-inspired case study on the adoption challenges
A Marchenko, P Abrahamsson - Agile, 2008. AGILE'08. …, 2008 - ieeexplore.ieee.org

Collaboration and co-ordination in mature eXtreme programming teams
H Sharp, H Robinson - International Journal of Human-Computer Studies, 2008 - Elsevier

The influence of organizational structure on software quality: an empirical case study
N Nagappan, B Murphy, V Basili - … conference on Software engineering, 2008 - dl.acm.org

Successful knowledge transfer within offshore supplier networks: a case study exploring social capital in strategic alliances. 

Rottman, J. (2008), Journal of Information Technology, 23, 31–43.

Distributed Communication as Collective Socio-material Sensemaking in Global Software WorkS Vidolov, S Kelly - 2009 - aisel.aisnet.org

Guidelines for conducting and reporting case study research in software engineering
P Runeson, M Höst - Empirical Software Engineering, 2009 - Springer

Understanding project survival in an ES environment: a sociomaterial practice perspective
EL Wagner, S Newell, G Piccoli - Journal of the Association for Information …, 2010 - Citeseer

Scrum+ engineering practices: Experiences of three microsoft teams
L Williams, G Brown, A Meltzer… - … Software Engineering …, 2011 - ieeexplore.ieee.org

Sociomaterial bricolage: The creation of location-spanning work practices by global softwaredevelopers
A Johri - Information and Software Technology, 2011 - Elsevier

Entanglements of creative agency and digital technology: a sociomaterial study of computer game development
NS Panourgias, J Nandhakumar… - … Forecasting and Social …, 2013 - Elsevier

The validity of case study research is often called into question by asserting that the findings from single cases cannot and should not be generalised beyond the specific contexts and events of the case's empirical setting. However, the criticism ignores our human ability to generalise and glean learning ourselves when we hear stories of others' experiences. You could call it vicarious learning, looking over someone's shoulder, it exemplifies our innate ability to juxtapose and translate learning from one context and apply it to a different context. Furthermore human activity systems, of which software engineering is but one example, are intrinsically 'in vivo' settings. They are open and transparent to  sociological enquiry insofar as its actors participate reflexively within a wider social world. The reflexivity of human actors is present in so far as we, as protagonists, are aware of and take account of ourselves in terms of our behaviour, actions, and agency.

I adopting a stance that human activity systems diverge from the usual notions of functional science and therefore argue for the value of learning gained from cases. Rather than extrapolating from the specific to the general, cases encourage us to reflect on and relate learning from the specifics of cases to other contexts or situations.

---

Footnote: The field you address will relate to the findings you arrive at from your analysis of the data you gather, which of course is influenced by the access you have to an empirical case study site.

empirical case + access levels -> data gathered -> analysis -> findings -> audience / field

The field or conference you publish in will have its own format template that is simply adapted for the write-up.

On the subject of Writing

A couple of general points may be useful.

Working Title: Initially, phrase a research question as the title of the paper (you can change it later).
Abstract: Restate and expand on the research question in the abstract (you can change it later when you have analysed your findings).

Research Access: Make good use of your personal access to your contacts, projects or companies, past or present for providing data.

Gather data and working backwards:
What I mean by this is that you will almost certainly end up changing/revising the research question as you go along, and the abstract will need to be revised at the end. The working title and abstract written at the beginning was just a first stab at the paper.
A research question is a prerequisite and precursor to a research project. Having a question puts the focus on a few things; the kinds of data you might expect to gather which could be anything from interviews, observations, documentary/documents, to literature review/desk research etc. etc.
A (tentative) research question usually implies particular kinds data, so ask, what kind of data does the question imply? What constitutes evidence for the phenomena under investigation? What data will (or might) provide the kinds of evidence that could be used to make justifiable statements about the research context?
A clear idea of the kind of evidence and data sought leads us to consider the types of data capture methods (research methods) that can be used to produce data or the evidence sought. This in turn indicates a typical overall research model, research design and research approach. Sometimes a programme of research will involve many of these approaches. For example, literature review, a distinctive family of desk based text research, is generally a prerequisite for all other research activities; evident in an introduction or positioning section of a paper, or constitute the whole research project itself. In very general terms, the gamut of research designs or models includes - but is not limited to:

Essay: Purely theoretical, elaborating a conjecture, speculation, conceptual, philosophical argumentation, thought experiment, word games, word play, rhetoric, logicism, formalism, constructionism.
Review, meta-analysis of prior research, selective literature review, systematic literature review.
Empirical: descriptive, idiosyncratic, case-study, naturalistic observation, questionnaire survey.
Correlational-causal: studies adopting systems model views, inputs factors, outcomes, case-control study, observational study, survey, structural equation modelling.
Statistical meta-analytic: research derived from other research, findings based on aggregations of other research.
Semi-experimental: research styled on intervention in settings, field experimentation not amenable to laboratory control, quasi-experiment, trials, living labs.
Experimental: classical closed system model of scientific discovery, reproducible experiment, blind experimentation, random assignment experiments.

(partially derived from the Wikipedia article on Research Design):

These various research models involve or require differing philosophical commitments and assumptions or beliefs around the nature reality and the world (worldview, ontology), and of the nature of knowledge (epistemology is the theory and nature of knowledge; objective, subjective, social). Usually it is sufficient to simply acknowledge the stance adopted for the purpose of the research, be it: interpretive, critical, critical realism, naive realism, post-modern, deconstruction, construction, positivist, aesthetic, utilitarian, speculative, qualitative, quantitative.

Improve the draft:

Commence your paper with an introductory/positioning piece that incorporates a short selective review of relevant recent literature critically addressing the topic areas you are working with.

Provide a presentation to peers or colleagues if possible, talking through your ideas, the data, your analysis and findings. Doing so will inevitably help refine how well you communicate the story, your message, convincing evidence, the key points, highlight your contribution and findings.

Template

Start using a scientific conference template for writing up. I recommend using the LaTeX or Word template from the ECIS 2015 conference (http://www.ecis2015.eu/participation/submissions.html)

Further reading

Links to various related conference templates below:
A LaTeX and a Word version of an ECIS Template - from Muenster 2015 - recommended!!
A LaTeX version of an ICIS Template - Ryan Schuetzler - a bit gnarly. A Word version of an ICIS Template - it's Word :-(

MIS40850 The Term Paper Guideline

MIS40850 The Term Paper Guideline

``A case study of software engineering. ''

 Write an 8 page research paper. This page count includes the References section. Graphs, images, tables are not included in the 8 pages and so are indicated by caption and cross-reference text only e.g. <<Table 1 goes here >> or << Figure 1 goes here>>. Copies of the actual tables, figures, images will go into the appendix after the 8 pages. There is no page limit on the appendix.

Please acknowledge and reference all third party material, copyright etc. University College Dublin policies on plagiarism apply.

Scope:

The paper is written in an academic style, including references, bibliography etc.

An original descriptive case study of organisational aspects of your work environment or dealing with a project that you are/were involved in, applying theories and ideas discussed in lectures to illustrate and critique the challenge of designing, managing, controlling various aspects of software development. Alternatively a desk investigation (e.g. focused literature review, industry data analysis) may be agreed.

The subject matter should be of interest to anyone who wants to find out more about current software development practices. Indicative topics or subject areas may include:

• Software methodology (how you would characterise it e.g. in-house, waterfall/staged, agile/XP/iterative)
• Technology stack (a list/functional view of actual tools used to design/code/build/release)
• Issue tracking (bugs, support, features etc.) How it relates to daily dev activities?
• Design approaches; do you have a distinctive approach to designing? Where is design done? Who by?
• Deployment; how you deploy builds or packages, or operate if a service. How customers get and use it. DevOps.
• Code reviews, bench checks, pair programming. Practices around code and design reviewing.
• Approach to testing, Unit Tests, TTD, how testing is done during dev, before a release, for each build etc.
• Build; your policy for build/release, i.e. how you package, version, stage.
• Feature request/development life cycle.
• Quality Management Systems. What management system is in place? How it relates to Software methodology?
• Team environment, numbers of people in typical team, typical daily interactions, how interactions captured?
• Management challenges: knowledge management, career development, maintenance versus new product dev.
• Software Process Improvement (differences if any between maintenance versus new product dev.)

Suggested outline structure

Abstract: A brief summary of the context and contribution in easy to digest words. 

Introduction and Literature: Use the introduction to set the stage and survey the literature. Explain/critique the current state of knowledge and drawing upon relevant/similar research publications.

Research method: A short statement of the research design and research methods employed.

Case description/context: A readable descriptive account of the salient features of an important period, project, episode based on your own development experience and your view of what it is/was really like (insider's view).

Analysis: Interpret a concise selection of representative case data findings. Employ relevant theories or models to help explain the findings. 

Discussion: Relate the findings to the literature, to current professional practice. Seek to generalise findings for wider application. 

Conclusions: An overall synthesis.

(the following are not included in the page count)

References: A bibliography of references cited in the report; use the template reference style or Harvard equivalent. 

Appendices: Diagrams, figures, tables, and additional relevant material may be included in appendices.

Grading Criteria

Grading will consider the following:

1. The research project (motivation and goals) is clearly explained.
2. Critical positioning in literature.
3. Empirical work, data and evidence presented.
4. Contributions are clear.
5. Overall quality of the document as a finished product.

And so; questions the examiner will ask when reading the paper will be:

• Is the research project (motivation and goals) clearly explained?
• How is it positioned in the literature?
• Is empirical work, data and evidence presented?
• Are the findings, conclusions, contributions clear?
• Overall quality, how does the document look and read?

A brief explanation of letter grade descriptors is provided below.

---

Modular (letter) grades.

A+/A

• The report is complete and covers all important topics.
• Appropriate significance is attached to the information presented.
• There is a compelling logic to the report that reveals clear insight and understanding of the issues.
• Analytical techniques used are appropriate and correctly deployed.
• The analysis is convincing, complete and enables creative insight.
• The report is written in a clear, lucid, thoughtful and integrated manner-with complete grammatical accuracy and appropriate transitions.

A-/B+

• The report is complete and covers all important topics.
• Appropriate significance is attached to the information presented.
• There is a clear logic to the report that reveals insight.
• Analytical techniques used are appropriate and correctly deployed.
• The analysis is convincing, complete and enables clear insight.
• The report is written in a clear, lucid, and thoughtful manner-with a high degree of grammatical accuracy.

B/B-

• The report is substantially complete, but an important aspect of the topic is not addressed.
• The report used information in a way that was inappropriate. 
• There is a clear logic to the report.
• Analytical techniques are deployed appropriately.
• The analysis is clear and the authors draw clear, but not comprehensive conclusions for their analyses.
• The report is written in a clear, lucid and thoughtful manner, with a good degree of grammatical accuracy.

C

• The report is incomplete, with important aspects not addressed.
• The report frequently used information that was substantially inappropriate or inappropriately deployed.
• The report’s analysis is incomplete and authors fail to draw relevant conclusions.
• The report is poorly written.

D

• The report is substantially incomplete.
• Whatever information provided is used inappropriately.
• There is little analysis and the report is inconclusive.
• The report is poorly written and presented.

Exercise: Writing for research

Using technology to manage a bibliography, referencing and in-text citations.
This exercise looks at MS Word style sheets and References...

1. Save a copy of the ECIS template to your own drive (ECIS-inspired-template.doc in Google Drive link).
2. Rename your file using the following pattern "Surname_MyResearchProject_2019.doc".
For example my own paper is saved as "HigginsEtAl_WorkingInVirtualLight_2019.doc". I have used the author convention "SurnameEtAl" as there are three or more authors.
3. From the MS Ribbon "Home" open the Styles Panel. The "Current style" field show the current text style wherever the cursor is in your Word file. Alternately navigate to top menu "Format>Style" for similar.

The current style at cursor location

4. Select section 1 "First level heading" and rename it "Introduction"

5. Paste and match formatting using following unformatted text as new paragraphs for section 1

Critical management studies appreciate that products and services, produced with technologies, by organisations, and the involvement of users, rely upon "actors having formal and symbolic resources for the exercise of... systematic forms of control over organisational participants, and indirectly over other groups and non-human objects"\citep{AlvDee2000aa}. The techniques and skill of management for producing digital goods and services (through software, hardware and systems) at its best aims to resolve this through the delicate, democratic balancing of power, control of resources, shaping of work culture, and leadership \citep{Kid1981aa}. The following brief introduction to the literature positions this study within the broad field of management information systems and seeks to inform further creative, design, and development initiatives.This study looks at...

6. Confirm that the paragraph current style is "Basic text"

7. Select the following text and change its style to "Subtle Emphasis". You many need to filter the style list selection at the bottom of the styles window.

“actors having formal and symbolic resources for the exercise of... systematic forms of control over organisational participants, and indirectly over other groups and non-human objects”

8. With the MS Ribbon "References" active...

Select the LaTeX citation command \citep{AlvDee2000aa} and replace it with the MS Word equivalent citation, i.e. from the References Ribbon select "Insert Citation". You may need to create a new entry in the Word file's Citations collection

Create a new citation source record

Enter a new source record as follows (n.b. add ", p. 7" to be thorough)

MS Word's new citation source editor

9. Similarly replace the LaTex/Bibtex command for Kidder "\citep{Kid1981aa}" with

Adding Tracy Kidder's Soul of a New Machine to the citation list in your MS Word document

10. Now regenerate the bibliography at the end of the draft paper by navigating to the "References" section, and selecting "Bibliography" to insert a new bibliography. You'll end up deleting the previous copy. You'll also need to reapply the style "Reference" to this text

n.b. the bibliography style-type (Harvard - Anglia) and the insert "Bibliography" command

Tips for comparative exercise - prep and presentation

Tell a good story...

What worked well? 

What didn't work well?

What would you do again?

What would you drop?

Was there or what was the division of responsibilities?

Could you explain how a typical day/week was organized?

***How are feature requirements justified by the business analysts?

Consideration of power, responsibilities, ownership. Who manages, chooses work?

What are the problems of 

Per product/service initiative...

   How many people...

   How much overtime...

   How many teams (team sizes and locations)...

Is everyone using Scrum style interactions (QA, Devops, Product management).

Where is architecture represented? Where do you manage architecture and design (although everything is design)?

What was the best part of working there?

What was the worst part of working there?

Offer a rich and detailed description of the practices on this team/project (not so much emphasis on structural or procedural facets).

Pointers on case research

An informative and instructive case study of even a small team can be very useful if the context is unique, extreme or polarised, and the research well designed and analysed.

On starting to position your research objectives:
Seek out other research papers dealing with similar issues (perhaps in other fields), research that has influenced your or that you might consider modelling your study on?

On starting to gather data:
Keep your thinking open ended...
Investigation is often driven by questions:
"what's distinctive or unique about the situation?"
"why?"
"what works, what doesn't work so well?"
"why?"
"what would you do again, what would you drop?"
"why?
And always try to establish corroborating evidence for claims, through documents and artefacts, by direct observation...

On starting to write up your data:
Consider including actual quotes or vignette style 'episodes' to illustrate important features, e.g. key moments, or pattern of regularity.

Use photographs, diagrams, sketches.

As a basic principal social and organisational research should not harm or disadvantage those involved. Therefore you will nearly always need to anonymise subjects identities. You may also need to anonymise your broader case identifiers too (company names, product).

Likewise it should not be necessary to expose any proprietary material, architectural detail, code or other intellectual data.

On starting to analyse, interpret and theorise:
Analysis should be in an impersonal style.

Commence interpretation against chosen theory or theories.

Start identifying relevant research literature to position your case study.

Suggest future directions for expanded study or more focused study.