Objectives: The aim of the project was to explore the theoretical constructs underpinning the discipline of health informatics and produce a cognitive map of the existing understanding of the discipline and develop the knowledge base of health informatics for the IMIA and the BCS.

Method: The five-phase project, described in this article, undertaken to define the discipline of health informatics used four forms of triangulation.

Results: The output from the project is a framework giving the 14 major headings (Subjects) and 245 elements, which together describe the current perception of the discipline of health informatics.

Conclusion: This article describes how each phase of the project was strengthened, through using triangulation within and between the different phases. This was done to ensure that the investigators could be confident in the confirmation and completeness of data, and assured of the validity and reliability of the final output of the ‘IMIA Knowledge Base’ that was endorsed by the IMIA Board in November 2009.

Subsequent aims of the project were to develop the knowledge base of health informatics, which was seen as central to the IMIA strategy (Murray 2008; Lorenzi 2007), and to undertake the task of exploring the current perceptions of the Health Informatics community as to the scope of the discipline.

Wolf (2010) in a recent article says the final consideration in using a mixed method approach is ‘to consider thoroughly whether to engage in triangulation, and if doing so, to use tailor-made triangulation strategies fitted to the research questions and interests’.

The project undertaken to define the discipline of health informatics used all four forms of triangulation (Denzin 1970) and this article describes how each phase of the project triangulated with the other phases for confirmation and completeness of data, and validation and verification of the project outputs.

Triangulation is a strategy to ‘overcome the intrinsic bias that comes from single methods, single observer and single theory studies’ (Patton 1990). Its objective is the confirmation and completeness of data through cross checking data from several sources to seek out consistencies in the data (Begley 1996; O’Donoghue & Punch2003). Many researchers also advocate triangulation as a means of resolving the quantitative and/or qualitative question through integrating the two approaches in one study and contributing to methodological rigor in order to validate the findings (Begley 1996; Cohen et al. 1994).

Denzin (1970) identified four forms of triangulation: data, investigator, theoretical and methodological. Data triangulation involves gathering data using different sampling strategies, so that segments of data are collected at various times, social situations and with different people. Investigator triangulation requires the use of more than one researcher in collecting and interpreting data. Using more than one theoretical position for data interpretation is called theoretical triangulation,whereas the most common form of triangulation, methodological triangulation, refers to the use of more than one method of data collection.

The ability to generalise findings to wider groups is one of the most common tests of validity for quantitative research. Triangulation is typically a strategy forimproving the validity and reliability of research findings. Patton (2002:247) advocates the use of triangulation stating ‘triangulation strengthens a study by combining methods’. However, the idea that triangulation is simply the combination of different methods of investigation is a restricted one, and researchers need

to increase their use of the other less frequently employed forms of triangulation. When using triangulation of methods, researchers should also reflect on whether the use of within-method triangulation would be advantageous to their project. Within-method triangulation involves using dissimilar aspects of the same method in one study; for example, a questionnaire might contain two different scales to measure emotions. Between-method triangulation involves using different research methods, for example a questionnaire and observation to collect data (Bryman 2003; Begley 1996). Sequential use of quantitative and qualitative methods may also be more effective for some projects rather than simultaneous use, which do not permit the development and refinement of the methodologies. The deliberate use of multiple data sources and methods to cross-check and validate findings, should pervade all projects and lead to the objective of confirmation. Triangulation should be chosen intentionally, and a description of its rationale, planning and implementation is essential in project reports to give authority to triangulation and the project outcomes (Begley 1996).

The project explored the theoretical constructs underpinning the discipline of health informatics. The early project work was situated within a theoretical educational framework. Bloom’s taxonomy affords a hierarchical scheme for categorising levels of complexity for objectives within educational settings (Bloom et al. 1984). It also overlays well against other academic levels, such as the progression from undergraduate to postgraduate levels (Furst 1981; Seddon 1978). Bloom classified three domains of educational activity (Forehand 2005):

This consensus conference therefore used group activities as its research methodology to produce lists of elements grouped into themes. There were six researchersinvolved in facilitating the group and plenary activities, thus adding investigator triangulation to reduce a single researcher bias.

After processing the data to count the number of occurrences of each keyword a master list of some 10 000 different keywords were identified, many of which were just English terms rather than health informatics specific, for example the authors place of abode and conference venue or country of study. The use of keywords inmany publications depends on author choice and often reflects the wish to have the article seen as being in a particular theme or subject area. This is particularly so with those conferences that identify themes for the submission of papers.

This activity produced a new set of data and so triangulated with phase one of the project that also produced raw data. In itself it was preparatory work for the next two phases of the project.

The participants in the workshop reduced the list of 10 000 words to 444. The number of occurrences found in the literature search ranked each keyword on the spreadsheet and small focus groups excluded words unconnected with health informatics. The remaining 444 words appeared to be connected with areas of health informatics as opposed to being just English words and phrases used to describe the content of the papers.

Voting in of keywords by international volunteers using a voting system based in an Excel spreadsheet

The voting was conducted with all of the keywords listed on the spreadsheet and a choice box next to each. The 444 keywords were divided into groups and each group was given a range of letters, A to G, H to M, N to R, and S to Z. Participants were asked to complete the group that contained the initial letter of their surname.Thus, as an example, Heather Carter voted on the columns A to G and Peter Ross voted on columns N to R.

Participants voted for about 100 words in their group. They were asked to vote for the keywords they thought were health informatics terms and classify themaccording to which phase one theme they thought the keyword belonged with by putting the number of the theme next to the word on the spreadsheet. Keywords that were consistently chosen were added to the original phase one cognitive map. These final two phases used methodological triangulation to refine the data and match it with the output of the first two phases: the phase one workshop and the international interpretation workshops.

The initial outputs from phase one have been used in a number of ways including to help formulate an undergraduate biomedical informatics degree programme (Pritchard-Copley et al. 2006) and as a framework to classify scientific papers for the European Federation for Medical Informatics (EFMI) conferences.