Communications in Information Literacy, Vol 1, No 2 (2007)

Font Size:  Small  Medium  Large
Author, Title, Communications in Information Literacy #(#)

Research Students in the Electronic Age

Impacts of changing information behavior on information literacy needs

Kirsty Williamson
Monash University and Charles Sturt University

Vivienne Bernath
Monash University

Steven Wright
Monash University

Jen Sullivan
Monash University

 

Abstract

Because of the rapid uptake of information and communication technology (ICT), understanding the ways in which information seeking has changed over the past decade is crucial to gaining a picture of how information literacy (IL) needs may also be changing in the electronic age. This qualitative research took an interpretivist/constructivist approach in examining the ways in which access to electronic information seeking affects the IL needs of 15 research students in an Australian university setting. An ethnographic technique, the interview, was used for data collection. Three particular areas related to information seeking and use were selected: (a) information source use, because of the burgeoning availability of electronic sources; (b) knowing when to stop collecting information, because the Internet has made greater quantities of information more easily available than in the past; and c) managing information following its collection, which has also been affected by the vast amount of information that is now accessible. The conclusion points to enhanced roles for both supervisors and academic librarians, with the need for the latter to become perceived as educators within their university communities.

Introduction

Undoubtedly, the availability of electronic access to information has gradually wrought major changes to human information behavior related to source use in all walks of life. This is certainly the case with students undertaking university studies. The central question of this article is: What are the implications of these changes for information literacy needs, with a particular focus on research students?1 Within this broad question, there are three particular areas of interest: the selection of sources, the transition from information gathering to information use, and the management of information. Another framing question will be: How can librarians and research supervisors help research students optimize their source choices, become more confident about when to move from searching to using the information which has been gathered, and learn better management skills related to their research?
           
There has been considerable exploration of information literacy in the educational sector in Australia. A key researcher, Bruce (1997), has emphasized that users “experience” information in different ways. This experiential approach fits well with the interpretivist/constructivist approach used for the research in this article.
            
Another key term is “information behavior.” Until the end of the twentieth century, this was not commonly used in the literature. Rather, the favored term was “information-seeking behavior.” It encompassed information needs, use of information sources, and information use following retrieval. As Williamson (1995) pointed out, attempts to discover preferences for information sources had figured prominently in studies of information-seeking behavior. The term now favored is “information behavior.” Fisher, Erdelez, and McKechnie (2005) conceptualized information behavior as “including how people need, seek, manage, give and use information in different contexts.” Since information seeking usually involves the use of information sources, source use (important to this article) is still encompassed in this definition. Knowing when to stop collecting information and managing information following its collection, other key foci of this article, are also encompassed in the Fisher et al. definition.
           
The elements of information behavior identified by Fisher et al. have also been used to define the attributes of information-literate people. Webber and Johnston (2006) provide an overview of key definitions of information literacy in which they use the term “information literates.” They also note wide reference to the American Library Association’s 1989 statement: “To be information literate, a person must be able to recognize when information is needed and have the ability to locate, evaluate and use effectively the needed information.” This definition appears in the American “Information Literacy Competency Standards for Higher Education” (Association of College & Research Libraries [ACRL], 2000) and in its Australian derivation, the Australian and New Zealand Information Literacy Framework (Bundy, 2004). The framework identifies six core standards as the basis of “information literacy acquisition, understanding and application by an individual.” The first standard includes the ability to know when to stop gathering information; use of appropriate sources is covered by the second and third standards; and the fourth standard explicitly addresses management of information.
         
Understanding how the deployment of information and communication technology (ICT) has changed the nature of tertiary learning over the past decade is crucial to trying to obtain a picture of how information literacy needs may be changing in the electronic age. Provision of study materials, staff–student communication, teaching practices, and models of learning are all being recast through the spread of ICT within the university sector (McCann, Christmass, Nicholson, and Stuparich, 1998). Such changes have been significant in the emphasis that library and information professionals are now giving to the meaning of information literacy, and its place within the learning process (Bundy, 2004). Indeed, the concept of literacy itself is being rethought (Snyder, 2002; Lankshear, Petters and Knobel, 2000; Selwyn, 1999). Not only is there a heightened need for critical approaches to online information retrieval, given the questionable authority of much information on the Web (Devlin, 1997; Kellner, 1998; Lee, 1999; Singh, 2001), but “an understanding of the relations among ideas is as important as, if not more important than, mastery of the ideas themselves” (Luke, 2000). Moreover, student use of more traditional learning materials requires further reflection in relation to the recent burgeoning of electronic sources.
           
A number of empirical studies focusing on undergraduates’ changing information behavior in the electronic age have been undertaken (see, e.g., Tenopir, Hitchcok, and Pillow, 2003). In contrast, with a few exceptions (e.g., Barrett, 2005; George et al., 2006; Heinstrom, 2002; Junni, 2007; and Macauley, 2001), there is little recent research that has focused specifically on research students. Barrett (2005) highlighted the paucity of research focusing on information-seeking behavior of graduate students, especially since the widespread influence of the Internet, but since that time, the research of George et al. (2006) and Junni (2007) has been published. The focus of both of these more recent studies is source use and information searching, the latter on the effect of the Internet on the type and quantity of information students’ use as references in master’s theses. Neither addresses questions related to the other two areas covered by this article.

Aims of the Research

In order to contribute to this still under-researched area, the authors set out to study Australian research students. They chose students from the Faculty of Information Technology (IT)2 at Monash University,3 believing that while these students are likely to be highly computer-literate and skilled users of electronic information, they may not have been exposed to effective methods of finding and evaluating information. While the research covered a wide spectrum of information-related questions4 (outlined in the “data collection” section below), this article focuses on the impact of the now wide availability of information in electronic format. The particular issues to be emphasized are: (a) the types of sources of information now being used by students (discussed under the heading “Source Use in the Electronic Age”); (b) how students know when they have collected enough information—an issue that appears not to have been considered with regard to research students (discussed under the heading “Knowing When to Stop”); and (c) how students manage the information collected (discussed under the heading “Management of Information”).

The first question, particularly, compares the use of electronic resources to use of print and personal information sources, together with views about, and preferences for, source use. The perceived authority of online sources is included in this discussion. The second question arises because the Internet has made greater quantities of information more easily available than in the past. A crucial question is how research students judge that they have “enough information.” According to Berryman (2006), this concept is beginning to be explored from different perspectives in the field of information-seeking behavior, but “there is still much we need to understand about what contextual influences shape the judgement of enough information.”  The third issue also relates to the greater quantities of information now available and concerns strategies for managing information in the electronic age.

From this point, the paper discusses further relevant literature, the philosophy and method, the findings of the study specific to the questions outlined above, and the conclusion, which includes a discussion of the implications for academic librarians.

Source Use in the Electronic Age

As mentioned above, there are three recent significant studies—George et al. (2006), Barrett (2005), and Junni (2007)—in which research students were the focus.
           
George et al. (2006) investigated the information-seeking behavior of 100 graduate students at the Carnegie Mellon Institute. Overall, the study found that graduate students’ information behavior is influenced by people. They preferred online sources, used the Internet and the library’s intranet to search online, and used print resources from their own and other university libraries. Factors influencing behavior included “convenience, speed and time restrictions; knowledge of services and sources; and course requirements.” An examination of disciplinary differences in use of sources revealed that the 14 computer science students reported the highest use of Google searches as well as searches for Web sites. They were also the least likely to question the quality of information found on the Internet. The groups with which the computer science students were compared were from art and architecture (16 students), business and policy (11), engineering (26), humanities (20), and sciences (13). Those from the humanities undertook Google searches least of all the groups, and searched for Web sites only marginally more than the lowest group (art and architecture students). This group was also the most critical of the quality of information found on the Internet.
           
The contrasting findings between computer science and humanities students are interesting in light of Barrett’s 2005 study of 10 graduate humanities students in Canada. Barrett found that “several participants described a generation gap in their departments, in that graduate students and younger faculty members tend to utilize electronic information technology far more that older faculty members.” The students saw IT as one of a variety of tools appropriate to their research (depending on the nature of the project). Several participants saw electronic information resources as readily available and “increasingly taken to be highly authoritative,” searchable databases as more efficient than print indexes, and remote access to full-text journals as convenient.  

Junni (2007) also found a difference among the students in the three discipline groups in her sample—economics, psychology and mathematics—with regard to the reference lists for their master’s theses, the number of Internet resources they used, how they sought and obtained publications, and how they selected their sources.  For example, psychology students used significantly more journal articles than economics students, who, in turn, used significantly more than mathematics students. Mathematics students, particularly, relied heavily on monographs and course literature. The implication was that this denoted a difference in Internet use, since “the Internet has not affected the availability of monographs or course literature. Rather the Internet has mostly increased the supply of available articles from scholarly journals.”
           
Numerous studies have indicated the crucial importance of interpersonal sources to all types of information seekers and topics. In the academic area, Mills (2003) discussed how university academics access personal sources for teaching and research information, while the survey by Heinstrom (2002) of 305 Finnish master’s thesis students found considerable use of informal information sources. In the latter study, teachers and supervisors followed books and journals as the most used sources of information, while fellow students were relied upon by nearly 40% of the sample, and friends by 25%. Indeed, some students mentioned people as their most "precious" information sources. 
           
Barrett (2005) and George et al. (2006) both confirmed that interpersonal sources are still crucial in the electronic age. Barrett found that his graduate student researchers had several forms of interpersonal contact, “providing ongoing support, guidance and feedback.” The supervisor was the most important contact. Other contacts were specialists beyond the student’s institution, fellow graduate students, conference attendees and librarians. George et al. devoted several pages of their article to key interpersonal sources: academic staff, fellow students, and university library personnel and other help.

Knowing When to Stop 

The problems arising from overabundance of information, particularly since the advent of the Internet, are widely discussed in the literature (see, e.g., Case, 2002; Allen and Shoard, 2005). Allen and Shoard cited Edmunds and Morris (2000) in saying that “there is a perception in the literature that information overload has been exacerbated by the recent rapid advances in information and communication technology.” Lyman and Varian (2003) found that “although the Internet is the newest medium for information flows, it is the fastest growing new medium of all time.” In 2003, the volume of information on the Web had at least tripled since 2000.  Moreover, they estimated that new stored information in a variety of formats, including print, grew about 30% a year between 1999 and 2002. Case (2002) discussed the possible consequences of information overload: the information selectivity or filtering that people undertake, the anxiety they may suffer, or the halt they may call to research when faced with an overwhelming amount of information, for example. 
            
On the other hand, the concept of “enough information” and how people determine when they should stop collecting information has received little research attention to date (Berryman, 2006). One of the exceptions is Limberg (1999), who used two descriptive categories, “information overload” and “enough information,” in her phenomenographic study of 25 Swedish high school seniors undertaking a task.  Kuhlthau (2004) raised the problem of what is “enough information,” calling it a “deceptively simple question” and exploring it in different work contexts. Despite the plethora of information now available to them, it seems to be a question that has not been addressed in relation to research students.
          
What theories in the literature can be used to shed light on the question of how research students might determine when they have “enough information”? One possibility is the concept of “optimal foraging,” where hunter-gatherers or animals adapt their behavior to survive. This concept was related to human behavior by Smith and Winterhalder (1992) and extended by Sandstrom (1994) and Pirolli and Card (1995) to help explain the environmental factors that influence humans’ information choices. “Information foraging” was applied by Pirolli and Card to “activities associated with assessing, seeking and handling information sources.” They emphasized, as did Sandstrom, the weighing of costs and benefits undertaken by information seekers. This idea can also be applied to the issue of “knowing when to stop,” where information seekers weigh up the cost in time or effort against the likely return to be gained from continuing the search.
           
The issue of time availability is crucial. Barrett (2005) found that decisions to concentrate more on writing up projects than seeking further information were very much affected by time constraints. He quoted one participant as talking about the clock running out. Other participants talked about having to “arbitrarily cut off” or reading “only what was crucial as the deadline approached.”

Management of Information

H. Bruce et al. (2004) identified several studies that have explored how people manage information in their daily lives or in their professions. They defined the goal of information management as “increas[ing] the likelihood that, whenever the information is needed, the individual will remember where it is and be able to re-find it.” Their own study investigated how information professionals, researchers, managers and students keep and re-find information from the Internet. The most popular method for researchers and for students was to save Web pages as “bookmarks” or “favorites.” Both of these groups next favored doing nothing about storing or recording information, but searching again when the information was needed. Level of use of personal information software varied from more than 25% of the researchers to only 10% of students.

Reflection on finding and managing information was a focus of a Swedish course designed to help PhD students develop their information literacy skills (Pilerot, 2004). Pilerot noted that doctoral students need to manage larger amounts of research information than other university students do. Although most of the 18 students in the case study were comfortable with their information searching and using skills, many used between 30 and 50 folders of articles and reference lists to manage their growing collections, despite half the group having had experience with personal information software before the course. Most students preferred printed versions of references, as they could be annotated easily. In the logs that assisted them in searching, managing, and using information “as a coherent process,” they revealed that they had problems handling large amounts of information.

An earlier study investigating information management skills of research students by Genoni and Partridge (2000) included supervisors as well as their students. Ten students from several humanities disciplines (including information studies) were at various stages of research ranging from early to near completion. The researchers considered development of advanced information management skills to be essential information literacy “in the context of higher degree research.” They found that early in the research process, few students had the ability to make the conceptual links necessary to organize their material well; few used electronic information management packages, or were aware of software features that would facilitate re-finding information when it was needed; and providing advice on information management was generally not seen as part of a supervisor’s role. The conclusion was that, despite student and supervisor expectations, “many students who undertake postgraduate research are poorly prepared for the personal research information management tasks which await them,” and even after some time in the research world, many students did not develop understanding and effective methods of handling the information they collected.

Research Philosophy and Method

For this study, the researchers adopted an interpretivist/constructivist approach in an attempt to understand the information literacy needs of research students, as well as the values, beliefs, and “meanings” they construct around the issues of information needs, information seeking, and knowledge integration. 
          
The study was undertaken with the approval of, and in compliance with, the procedures deemed appropriate by the Monash University Standing Committee on Ethics in Research Involving Humans (SCHER).

The Sample
Fifteen students were purposively selected using a limited form of theoretical sampling which did not extend, due to time constraints, to returning to the field to fill conceptual gaps and holes (Charmaz, 2003). First introduced by Glaser and Strauss (1967), the concept of theoretical sampling involves the selection of participants who represent the major categories of people relevant to the research. With theoretical sampling, there is no compunction to sample multiple cases which do not “…extend or modify emerging theory” (Henwood and Pidgeon, 1993). In this case, the major category was “students undertaking a research degree,” with type of degree and place of study for undergraduate degrees (Monash University or elsewhere) being considered as subcategories, and gender and age being of some (though limited) importance. The researchers decided to include only research students from one faculty (Faculty of Information Technology at Monash University) so that the sample was relatively homogeneous for other key dimensions. It would have been interesting to have selected students with diverse backgrounds, but because of the necessarily small sample, it was felt that points of consensus on key issues would be difficult to obtain when comparing, e.g., humanities students with IT students. The literature indicated that these two groups, particularly, would be quite different in their needs and skills.
           
To obtain the sample, lecturers made the project known to their students.  The sample included two Honours,5 three Research Masters and 10 PhD students, of whom nine were female and six male. Six were aged in their 20s, seven were in their 30s, 40s, or 50s, and two were 60 or older. Nine students had gained their undergraduate degrees from Australian universities. Of these students, four had studied at Monash University. Six students had undergraduate degrees from non-Australian universities.   

Data Collection
An ethnographic technique, the interview, was used for the data collection.  The initial step was to develop a semistructured interview schedule. All four team members were then involved in piloting and repiloting the interview schedule. The final schedule included 11 questions, some with prompts so that data were not missed if particular points were not spontaneously mentioned by interviewees. The questions ranged across topics such as selection, defining, and redefining the research topic; sources of information; knowing when sufficient information has been collected; the use of, and getting help with, online resources; determining the authority of online resources; the management and assimilation of information; the role of previous study and experience; and the ways in which information seeking could be improved, including the role that librarians might play. 
           
With regard to the individual interviews, all four team members, in different combinations of two, took turns conducting the interviews, which lasted about one hour. With the permission of the participants, the interviews were audiotaped.

Data Analysis
The audiotapes of the interviews were transcribed by an experienced transcription typist. Although the analysis as undertaken does not constitute a “grounded theory,” it was influenced by the “constructivist grounded theory” approach of Charmaz (2003). Charmaz says that, unlike the original grounded theory (Glaser and Strauss, 1967) and particularly the later version written by Strauss and Corbin (1990), constructivist grounded theory is not “objectivist.”  It “recognises that the viewer creates the data and ensuing analysis through interaction with the viewed” and therefore the data do not provide a window on an objective reality. Charmaz, therefore, recognizes that researchers’ backgrounds will influence their interpretations of the data. They cannot avoid being influenced by “disciplinary emphases” and “perceptual proclivities.” This means that although every effort is made to look at “how ‘variables’ are grounded—given meaning and played out in subjects’ lives” (Dawson and Prus, 1995; Prus, 1996, as cited by Charmaz, 2003), there is acceptance that “we shape the data collection and redirect our analysis as new issues emerge”(Charmaz, 2003).

All four researchers were involved in the analysis of the data, initially independently. They made margin notes on individual transcripts, highlighting words they thought would be potential themes or categories within themes. At this point they compared their analyses and found there was almost total agreement about the main themes. Passages of data were categorized and linked to one of the themes.  Examples of themes, categories, and related quotations are presented in Table 1.

Table 1 -- Examples of themes, categories and quotations: "Knowing when to stop" issue

Themes

Categories

Quotations

Feelings of information overload Reaching saturation point …you get to a saturation point, I guess, where you have 400 or so references
Not  knowing when to stop

I don't know the cut off point and I guess I'll just keep reading…

 

Strategies used for deciding when enough information had been collected Looking for redundancy (repetition) of information It's when they start repeating and nothing new is coming.
Reliance on the supervisor

…that's something you use your supervisor for to say “That's enough.”

 

Starting the writing process

I don't think I know I have enough information.  What helps me is when I start writing …

 

 

Findings

Participants were asked about various aspects of their study and research, from the initial definition of a research topic to the seeking, organizing, and using relevant sources. The researchers also inquired about previous study and experience; the role that  supervisors played in students’ information use and management; and the ways in which information seeking could be improved, including the role that the library might play. In this article, the discussion focuses on three specific topics: research students’ use of sources of information, how they determine when they should stop collecting information, and how they manage the information they have collected.

Use of information sources
The researchers’ findings indicated that the Internet and other electronic search tools had exerted considerable influence on the way participants in the study searched for information, and had an impact on the kinds of sources selected. In keeping with a recent study of a broad population of U.S. tertiary students (Online Computer Library Center [OCLC], 2002), the researchers found that search engines such as Google were popular among students in the sample, even when they were aware of, and used, library-provided databases. Indeed, given its familiarity, speed, and a large number of potential “hits,” Google was for many participants the yardstick against which other search tools were assessed, as comments such as the following attest:

I think our library database search engine isn’t as good as it could be. Sometimes I’ll put in a keyword and it will come up with a whole heap of stuff that isn’t very relevant whereas Google would give me [something] more relevant …You’ve got better chances of finding something.

This student also said: “I didn’t realize until recently that the library had a lot of electronic papers online and all of that.”

Not all information used by participants was electronic. For many, the question of what kind of source to use in their research was less one of media form (digital or non-digital) than of the nature of the documentary formats predominant within their disciplines. As the studies by George et al. (2006) and Junni (2007) found, disciplinary differences could help shape the worth and accessibility of sources for participants. For example, some of the subdisciplines clustering in and around information technology placed differing weights upon the relative worth of books or book chapters, compared to journal articles or conference papers. For those students who needed access to the latest research findings in their field, even electronically published journal articles could sometimes be considered too slow in terms of keeping up with the “cutting edge” of debate—and hard copy books even more so. In such circumstances, one student argued, the best source of relevant materials took the form of working papers or draft conference papers, often available from an individual academic’s personal Web page.

While most participants were conscious of the need to find information appropriate to the topic, regardless of media form, there were participants whose passion for electronic information knew no bounds. One Honours student informed us that “it’s great having the Internet because you can find almost anything.” Another, having decided in high school that it was “very annoying going through books,” had a strong and almost exclusive preference for online materials. More typical of those interviewed were students who relied heavily upon electronic sources, both from the wider Internet and from academic journals to which the university subscribed. 

The strongest argument in favor of online sources was convenience. As one student put it, “I’ve got a library at home because I’ve got a computer terminal.” Through online access, materials could be downloaded and printed out around the clock; in such cases there was no need to travel to campus, let alone queue to use photocopiers. And with many students engaged in paid work, being able to fit study into the home/work routine was of crucial significance:

Because I’m working full time now and I was working part time before, it’s not always possible to get to the library to go through the catalog, but I can go through the catalog from home. I can also use some of the search engines to find leads that I wouldn’t find any other way … Because I’m working full time, a lot of the reading I do is either in the early hours of the morning or the very late hours of the night.

One of the most interesting findings of the study was that a number of the students interviewed did not necessarily make hard and fast distinctions between different kinds of electronic sources. Instead, they appeared to conflate, under the general rubric of “online,” both academic journals accessible through the library and a host of sites found through Google. This raises the matter of the authority of online sources, a topic that offered considerable variation in responses. On one hand, an Honours student suggested, “most of my research has come from people who work at universities or who are lecturers or have some sort of tertiary qualification. So because of that I don’t really question the reliability of the source.” On the other hand, a few participants insisted upon the academic peer review and citation as one crucial filter in this regard: “The information I’m gathering has gone through peer review processes.”

As other studies have found, people are still important to students in the electronic age. The supervisor was a key resource for most participants, although not all. At one extreme stood a student whose supervisor was of little importance in producing an Honours thesis; indeed, she recalled, “I hardly spoke to her.” Nor were the thoughts of other academic staff or fellow students deemed relevant, with the student choosing instead “to keep to myself.” Then again, the other Honours student drew attention to the input provided by her supervisors, particularly in the structuring and design of the thesis itself. Even students who appeared competent in finding their own information still acknowledged the role of supervisors in helping to provide a framework for the research project, as well as an ongoing reference point and sounding board for their work. For example, one student emphasized the help provided by a supervisor in “narrowing down” a PhD research topic when she had become “a bit lost” after “going through many different things.” A student working in the field of information management reported that she would sometimes ask her supervisors for leads concerning information sources, while also turning occasionally to online forums. Another participant mentioned the usefulness at times of “bounc[ing] ideas off” other research students, both informally and within the context of a research methods class. Academic staff were crucial to this student too, above all for their experience, which allowed them to “point out to me where I’ve missed” aspects of intellectual debates.

Knowing when to stop gathering information
Researchers, whether seasoned or novice, have real difficulties in determining if they have accrued sufficient relevant information. As Kuhlthau (1991) pointed out, bringing information “collection” to an end is often bound up with providing “focus” to a project, something that is not easy in research. With the increased availability of information in the electronic age, the task of “knowing when to stop” has undoubtedly become even more of a Sisyphean task.
           
Participants were asked how they knew if they had gathered enough information. A majority said that they did not know when to stop the information searching phase of their research. They made comments such as “it is very difficult” or “you get to a saturation point I guess where you have 400 or so references but you never know whether you've missed something.” Another student expressed his apparent dismay at the unending aspect of the information search in his research area: “I don't know the cut-off point and I guess I'll just keep reading until I finish the project. The literature review won't finish until the day I put the final full stop on the thesis.” He continued to ponder how widely he should read in relation to peripheral information: “And one of the questions I have is the limit on how widely I read too, because it is all very well to research the problem but there are all the peripheral things I could draw in.” The sense of information overload, as discussed by Case (2002) is evident in these comments.
           
Strategies students used to help them to know when to stop included looking for repetition of information. As one participant said, “It’s when they start repeating and nothing new is coming.” Another participant reported asking her supervisor about when she would know when to stop searching and had been told that it is “when you start to read the same thing.” Limberg (1999) found that several of her Swedish students mentioned this approach.
           
Other students also used their supervisors’ advice as they contended with information overload, one stating “that's something you use your supervisor for: to say ‘that's enough.’” Students deep in the research process clearly found it useful to have a mentor who could apply the brakes of objectivity on a search phase that might be ballooning out of control. One student whose supervisor told her, “You're very good at gathering data but not so very good at writing up,” commented that “that's always a very strong nudge in the ribs.” 
           
There were other approaches as well. One participant felt that starting the writing process definitely helped in knowing when to stop searching: “I don't think I know I have enough information. What helps me is when I start writing … I have to be able to write it down for me to know what I'm thinking.” Another student’s approach was to set a date for completion of the search phase of research. While this might appear mechanistic, this could also be an example of the weighing of costs and benefits as discussed by Sandstrom (1994) and Pirolli and Card (1995).  In the student’s words:

That's very difficult, very difficult.  At the moment I've made a decision that it's going to be June 2003 and I'm not reading any more until I've written. And then that can make it lose its shape a bit. In the end you have to write and it has to have some sort of coherence to it.

Another example of this kind of approach is evident in the following quote:

You read more and you read more … I've read 10 or 20 books that tell the same thing … so I'm looking for the definition and I am not quite satisfied. But now I have to tell myself after 10 books, ‘Enough books.’ Otherwise it is neverending.  
           
Yet again a participant, researching in a technical area with a dearth of published literature, made the decision to draw a line under his search efforts, and move on to the next question or phase of research:

Once I feel I've got a reasonably sufficient and satisfactory answer then I just move on. Then maybe six months down the track I might just happen to come across another paper that is actually relevant too and I might go back and try and incorporate it.
 
This approach was similar to that of other students who felt they had a good understanding of their topics, and had enough evidence to answer each question, and so had an ability to stop searching: “If you feel you can give enough references and got enough evidence for something you're writing for your thesis I think that would be the place I can stop.” Limberg (1999) also found that this was an approach used by some of her students.
           
A student who also set up timelines and a structured approach for himself found that talking to other people in the relevant research area can provide a useful indicator as to whether enough information has been gathered:

I set myself three months for searching for information … One day I would do the search, another day I would do all the reading … I just try my best to retrieve all the work that has been done in this area. That's why I say it is not much … Otherwise I try to talk to people.

Similarly, one of Macauley’s (2001) participants noted that working alone too much could lead one to “re-invent the wheel.”
           
Barrett’s (2005) findings lead to the expectation that students would be very concerned about the deadlines they faced. While there was some mention of the finite nature of the time available, e.g., “You are doing something for the PhD in a limited amount of time so you have to know when [to stop] … and write the things formally for your thesis,” this issue did not arise as often as expected.

Management of Information
Genoni and Partridge (2000) noted that research students are “faced with far more challenging tasks in terms of storing, structuring, collating and recalling … information” than has been the case in their undergraduate or learning phase of information seeking. Management of information is a perennial problem for research students, allied to knowing when to stop searching. As the amount of material collected grows so does the need for a reliable method for organizing it.
           
The study participants’ abilities in this area ranged from a fairly formal organizational approach to trust in memory as a method of storage and retrieval. As one participant said, “The easiest, I find, is to just keep it all in my head, and most of the time I will remember.” This is in line with Genoni and Partridge’s (2000) findings related to research students’ awareness of information management issues.
           
On the other hand, more than half of our participants used EndNote or other software tools for electronic management, including entering short summaries of content. This is in keeping with the higher level of use of personal information software by researchers cited by H. Bruce et al. (2004), mentioned above. One student developed his own database for keeping track of his material. “I print out or photocopy all the articles. I index them. I have a little Access database which I key in the titles and keywords and all the authors and then I can do cross-referencing of the authors to see if they've been cited in other papers.”

This quote highlights another quirk of searching in the digital age. Because of their need for easier reading than is provided by a computer screen, most students made paper copies of information they had obtained electronically. This led to the need for methods of both electronic organization and for physical storage, the latter often being stacks of paper on the floor, with sticky notes attached giving brief summaries of specific papers. A variation on this situation was bemoaned by one of the students: the task of keeping her online folders synchronized with her hard copy folders. In fact, she was finding it easier to locate hard copies at a particular stage of her research. Another aspect of the need that was felt for both print and electronic versions is reflected in the words of the student who described the downside of online searching for her: “It is a kind of problem with online searching that it is more time-consuming in that you look at the paper, you think that it is okay, you downloaded it, you printed it and then when you're reading it is not much help.”

There was one complication that can easily arise for research students: the changes in focus that often occur in the earlier stages of a research project for a higher degree. One participant alluded to difficulties in reorganizing materials when he faced that situation: “The articles changed as the nature of the topic changed—I've discarded great lumps of documentation and put in new stuff. I've got a categorization I'm not completely happy with and I'll possibly recategorize.”

Discussion

The three areas highlighted in this article are all linked to human information behavior and information literacy in the electronic age. The authors have confirmed the findings of other studies that have indicated that the online environment is now very important to research students. While the authors were unable to take a comparative approach, and could focus on students studying in one faculty only, albeit with some differences in subject matter playing a part, some comparative data is available from George et al. (2006) and Barrett (2005). From these studies, it seems likely that the students in this study would have been higher users of electronic sources than comparable research students studying humanities at Monash University at the time of the study. For example, George et al. (2006) found that the computer science students in their study reported the highest level of Google searches (93%), compared to 50% by humanities students, while Barrett found that, although there was increasing use of electronic sources among his 10 graduate humanities students, the students saw information technology as one of a variety of tools appropriate to their research, depending on the nature of the project. In this study, despite their frequent use of electronic resources, many students attempted to use the most appropriate sources available. On the other hand, other students (e.g., the two studying for Honours) not only used electronic sources almost exclusively, but also appeared undiscriminating in their use of them.
           
A startling finding made by George et al. was that not one of the 14 computer science students in their study spoke about the possible questionable quality of information found on the Internet (compared to 30% of humanities students). While there were variations among this study’s IT sample with regard to the issue of judging the authority of electronic sources, it is also clear that not all research students took a critical, evaluative approach to electronic information. This was particularly the case with the Honours students, both of whom were young and relatively inexperienced compared with the master’s and PhD students. Related to this is lack of distinction between various kinds of electronic sources in the minds of some students. 
           
The results of Chapman (2002), who surveyed a cross-section of undergraduates, postgraduates and academic staff in one Australian university, confirm these findings. The last of her three “skills-based problems,” experienced across all user types regardless of experience or level of skill (including academic staff) was “inability to identify and select authorised [sic] information.”  She described “the reluctance of many unskilled and unsuccessful Internet searchers to give up time to undertake training to become efficient and effective users” as “one of the more disturbing issues arising from the research.”
           
It is interesting to note the continuing importance of personal sources of information in the electronic age.  For as long as information-seeking behavior/information behavior research has been undertaken, the role of people as sources of information has continually emerged.
           
The question of “knowing when to stop” seemed a vexing one for the students in this study, many of whom prefaced their responses to our question with phrases such as “that’s difficult” or “I don’t know”—although they then went on sometimes to talk about strategies that they had or could have used. Since it seems that this issue has not been previously explored with research students, and is only beginning to be examined closely with other groups, the authors do not have comparative findings.  Nevertheless, ways of assisting students to judge when to stop the collection process will become even more crucial as the amount of available information burgeons. This issue deserves wider investigation. In the meantime, the following is a summary of the main strategies used by the participants in this study: looking for repetition of information (redundancy); the advice of the supervisor; beginning the writing process; setting a date for the completion of the search phase of the research; gauging that there is enough information to answer a particular question; and talking to other people who might help “the reinvention of the wheel.” A looming deadline will often provide the incentive to stop the search!
           
Like Genoni and Partridge (2000), the authors believe that development of advanced personal information management skills is essential for information literacy in students working toward higher degrees. While a higher proportion of students in this study used software tools for electronic management than Genoni and Partridge found, they were not always aware of the full capabilities of the software. Reliance on memory was a key strategy for information management all too often. Since most students still set considerable store by having print copies available to them, there is also the problem of managing the print versions along with the electronic. This parallel management of electronic and hard copy documents is hardly a problem unique to researchers, and is a common challenge facing many organizations today. While personal information management has become a topic of growing interest among information professionals, more work needs to be undertaken in examining how the issue is dealt with in practice. If there is unlikely to be “one best way” for research students to address the question, the fact remains, as Marshall and Jones (2006) pointed out, that “[a] good match between how something is kept and its envisioned role or function is essential for using the material effectively and enjoyably.”

Conclusion

What are the implications of the study for assisting research students with information literacy in the electronic age? The authors suggest that both librarians and supervisors can do more in all three areas discussed in this article, but that they may not necessarily share equally in the opportunities in each case. For example, supervisors will have greater opportunity to provide research students with strategies for “knowing when to stop” than librarians, but the latter need to be aware of this issue and include advice on strategies as part of any information literacy instruction (ILI) tailored to research students. The list of strategies suggested by participants in this research may provide a starting point.
           
Because of their training, librarians are in a strong position to assist researchers with selection of information sources and management of information, as well as other components of information literacy. If information literacy is seen as a responsibility of the whole institution, policy initiatives can support librarians in their efforts to make students and academic staff aware of their specialized skills. The authors see institutional approaches as essential if the expertise available in academic libraries is to be used to advantage in promoting information literacy.
           
Chapman (2002) concluded that the reluctance of her research participants to undertake training in electronic source use “encourages the library to collaborate with academic teaching staff to ensure the training is included in the curriculum” and “to promote the training effectively by emphasising the advantages and efficiencies to be gained.”  The authors endorse those views. Librarians need to become perceived as educators within their university communities so that this involvement in the curriculum can take place. Their inclusion in the induction of new academics is a vital step. They must also make the effort to share what they learn about the needs of research students within their organizations and through publishing their research results.
           
In a recent article focusing on the “affordances” offered to graduate students at the Library of the University of Alberta (Sadler and Given, 2007), the researchers concluded that librarians were using ILI and the Web site almost exclusively to communicate with their graduate students. The study, in fact, indicated that participants were not aware of ILI, did not read notices on the library home page, and that personal contact of graduate students with librarians is “possibly the most effective tool the academic library has at its disposal.” Sadler and Given noted the difficulty of this, but concluded that “academic librarians must focus their energies on promotional dialogue with faculty and students.”
           
The message from the Sadler and Given research is that, in promoting ILI through the curriculum (as the authors recommend), librarians need to communicate directly with supervisors and their research students, attempting to include the former in their programs as well as the latter. In this way, supervisors will improve their own information literacy skills and are more likely to use the opportunities offered by their contact with their students—not only to point them toward the library, but to give them direct assistance in areas vital to the success of their studies.

It is essential that the importance of information literacy continues to be promoted so that it becomes more widely recognized as an essential graduate attribute within tertiary education institutions. It will then become accepted policy to incorporate information literacy education more explicitly within the general curriculum. Students will be more aware earlier in their studies of the need to select appropriate sources of information, to evaluate information no matter where it has been obtained, and to develop good practice in managing information.

Studies such as the one reported here are useful in highlighting the particular needs of research students. Staff development programs for academic staff who supervise research students can be enriched by raising awareness of the students’ information literacy development needs and discussion of ways to address them. Librarians’ involvement is recommended in these programs and in those aimed at assisting research students to develop skills which will help them make the most of their research experience.

References

Allen, D. K., & Shoard, M. (2005). Spreading the load: Mobile information and communications technologies and their effect on information overload. Information Research, 10 (2), Art. No. 227.

American Library Association (1989). Presidential committee on information literacy: Final report. Retrieved July 16, 2007, from http://www.ala.org/ala/acrl/acrlpubs/whitepapers/presidential.htm.

Association of College & Research Libraries (ACRL). (2000, last revised July 2006). Information literacy competency standards for higher education. Retrieved July 16, 2007, from http://www.ala.org/ala/acrl/acrlstandards/informationliteracycompetency.htm.

Barrett, A. (2005). The information-seeking habits of graduate student researchers in the humanities. The Journal of Academic Librarianship, 31 (4), 324-331.

Berryman, J. (2006). Judgments during information seeking: emerging themes from an exploratory study into the concept of enough information [Electronic version]. In A. Lloyd & B. Pymm (Eds.), Research Applications in Information and Library Studies Seminar (RAILS 2): Proceedings of the 2nd Research Applications in Information and Library Studies Seminar. Retrieved July 16, 2007, from http://www.csu.edu.au/faculty/educat/sis/
CIS/epubs/RAILS2/RAILS2_Judgements_info_seeking.pdf.

Bruce, C. (1997). Seven faces of information literacy in higher education. Retrieved July 16, 2007, from http://sky.fit.qut.edu.au/~bruce/inflit/faces/faces1.php.

Bruce, H., Jones, W., & Dumais, S. (2004). Information behaviour that keeps found things found. Information Research, 10 (1), October 2004. Retrieved January 25, 2007, from http://informationr.net/ir/10-1/paper207.html.

Bundy, A.(Ed.), (2004). Australian and New Zealand information literacy framework: Principles, standards and practice. Adelaide: Australian and New Zealand Institute for Information Literacy.

Case, D. (2002). Looking for information: A Survey of research on information seeking, needs, and behavior. Amsterdam: Academic Press.

Chapman, L. (2002). Russian roulette or Pandora's box: Use of Internet as a research tool. Paper presented at VALA 2002: 11th Biennial Conference and Exhibition, 6-8 February, 2002, Melbourne. Retrieved April 22, 2007, from: http://www.vala.org.au/vala2002/2002pdf/18Chpmn.pdf.

Charmaz, K. (2003). Grounded theory: Objectivist and constructivist methods. In N. K. Denzin & Y. S. Lincoln (Eds.), Strategies of qualitative inquiry (2nd ed.). Thousand Oaks, CA: Sage.

Devlin, B. (1997). Conceptual models for network literacy. The Electronic Library, 15 (5), 363–368.

Fisher, K E., Erdelez, S., & McKechnie, E. F. (Eds.). (2005). Theories of Information Behavior: A researcher's guide (pp. 128–132).  Medford, NJ: Information Today.

Genoni, P, & Partridge, J. (2000). Personal research information management: information literacy and the research student. In C. Bruce and P. Candy (Eds.), Information literacy around the world: Advances in programs and research (pp. 223-236). Wagga Wagga, NSW: Centre for Information Studies, Charles Sturt University.

George, C., Bright, A., Hurlbert, T., Linke, E.C., St. Clair, G. & Stein, J. (2006). Scholarly use of information: Graduate students' information seeking behaviour. Information Research, 11 (4), Art. No. 272. Retrieved July 16, 2007, from http://InformationR.net/ir/11-4/paper272.html.

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory.  Chicago: Aldine.

Heinstrom, J. (2002). Fast surfers, broad scanners and deep divers: Personality and information-seeking behaviour. Åbo (Turku), Finland: Åbo Akademi University Press.

Henwood, K., & Pidgeon, N. (1993). Qualitative research and psychological theorising. In Hammersley, Martyn (Ed.) Social research: Philosophy, politics and practice. London: Sage.

Junni, P. (2007). Students seeking information for their Masters' theses: the effect of the Internet. Information Research, 12 (2). Retrieved April 22, 2007, from http://InformationR.net/ir/12-2/paper305.html.

Kellner, D. (1998). Multiple literacies and critical pedagogy in a multicultural society, Educational Theory, 48 (1), 103–122.

Kuhlthau, C. (1991). Inside the search process: information seeking from the user's perspective. Journal of the American Society for Information Science, 42, 361–371.

Kuhlthau, C. (2004). Seeking meaning: A process approach to library and information services. 2nd ed. Westport, CT: Libraries Unlimited.

Lankshear, C., M. Petters & M. Knobel (2000). Information, knowledge and learning: Some issues facing epistemology and education in a digital age. Journal of Philosophy of Education, 34 (1), 17–39.

Lee, A. (1999). Infomedia literacy: An educational basic for young people in the new information age. Information, Communication and Society, 2 (2), 134–155.

Limberg, L. (1999). Experiencing information seeking and learning: a study of the interaction between two phenomena. Information Research, 5 (1).  Retrieved January 20, 2007, from http://informationr.net/ir/5-1/paper68.html.

Luke, C. (2000). Cyber-schooling and technological change: Multiliteracies for new times. In B. Cope, & M. Kalantzis (Eds.), Multiliteracies: Literacy Learning and the Design of Social Futures. Melbourne: Macmillan.

Lyman, P. & Varian, H. R. (2003). How much information, 2003. Technical report, UC Berkeley. Retrieved July 16, 2007, from http://www.sims.berkeley.edu/research/projects/how-much-info-2003/.

Macauley, P. (2001). Doctoral research and scholarly communication: Candidates, supervisors and information literacy. PhD Thesis, Deakin University, Geelong: Faculty of Education.

Marshall, C. C., & Jones, W. (2006). Keeping encountered information. Communications of the ACM, 49 (1), 66–67.

McCann, D., Christmass, J., Nicholson, P., & Stuparich, J. (1998). Educational technology in higher education. Canberra: Department of Education, Training and Youth Affairs.

Mills, J. (2003). Information seeking behaviour of university academics. PhD thesis. Wagga Wagga: Charles Sturt University.

Online Computer Library Center (2002). OCLC white paper on the information habits of college students. Retrieved April 26, 2007, from http://www.oclc.org/research/announcements/2002-06-24.htm.

Pilerot, O. (2004). Information literacy education for PhD-students—a case study. Proceedings of the 12th Nordic Conference on Information and Documentation 2004, 92–98. Retrieved January 25, 2007, from
http://www2.db.dk/NIOD/pilerot.pdf.

Pirolli, P., & Card, C. (1995). Information foraging in information access environments. CHI ’95 Proceedings. Retrieved July 16, 2007, from http://www.sigchi.org/chi95/Electronic/documnts/papers/ppp_bdy.htm.

Sadler, E., & Given, L. (2007).  Affordance theory: A framework for graduate students' information behavior. Journal of Documentation, 63 (1), 115–141. Retrieved April 22, 2007, from http://www.emeraldinsight.com/Insight/viewContentItem.do?contentType=Article&contentId=1589320.

Sandstrom, P. E. (1994). An optimal foraging approach to information seeking and use. Library Quarterly, 64, 414–449.

Selwyn, N. (1999). Schooling the information society? The place of the information superhighway in education. Information, Communication and Society, 2 (2), 156–173.

Singh, M. (2001). The new technologies informing student interaction, knowledge production and reflection: The changing practices of language and literacy education. Keynote address presented at the NMIT Faculty of Further Education Conference, 16-17 July.

Smith, E. A., & Winterhalder, B. (1992). Evolutionary ecology and human behavior. New York: de Gruyter.

Snyder, I. (2002). Communication, imagination, critique—Literacy education for the electronic age. In I. Snyder (Ed.), Silicon literacies: Communication, innovation and education in the electronic age. London: Routledge.

Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory, procedures and techniques.  Newbury Park, CA: Sage.

Tenopir, C., Hitchcock, B., & Pillow, A. (2003). Use and users of electronic library resources: An overview and analysis of recent research studies. Washington, DC: Council on Library and Information Resources. 

Webber, S. & Johnston, B. (2006). Information literacy: Definitions and models. Retrieved July 16, 2006, from http://dis.shef.ac.uk/literacy/definitions.htm.

Williamson, K. (1995). Older adults: Information, communication and telecommunications. Unpublished doctoral dissertation, Monash University, Melbourne.

Wright, S., Williamson, K., Bernath, V., & Sullivan, J. (2006). The information literacy needs and practices of research students in the digital age [Electronic version]. In A. Lloyd & B. Pymm (Eds.), Research Applications in Information and Library Studies Seminar (RAILS 2): Proceedings of the 2nd Research Applications in Information and Library Studies Seminar, Retrieved July 16, 2007, from http://www.csu.edu.au/faculty/educat/sis/CIS/epubs/RAILS2/RAILS2_Info_literacy.pdf.

Notes

  1. The term “research student” has been used in preference to “graduate student” because, in Australia, many master’s students do coursework only, and the authors’ intention was to focus on students who, as with PhD students, undertake research and are then assessed solely or primarily on the basis of the thesis they have produced.
  1. In Australia, the word “Faculty” does not refer to full-time teaching employees as in America. Rather, it refers to the larger administrative group which brings together a number of departments, or schools as they are often called, from related disciplinary areas.
  1. The research was funded by a Monash University Small Grant. 
  1. A paper briefly discussing all of the issues covered by the research has already been published (Wright, Williamson, Bernath, and Sullivan, 2006).  Additional issues included in that paper, but not covered in the present articles, are selection of research topics and online search tools and search strategies.
  1. Although Honours students are classified as undergraduates in Australia, they must undertake research and write a thesis (unlike coursework master’s students).  This means that it is appropriate to classify them as research students.