Where did the time go? A short post about Rapid Ethnography

Well, it happened. I let over six months go by with nary a blog post! Ack! And the thing is, I have been so busy, with lots of exciting UX stuff going on. So, I should have been writing about it all along. So, if you are still out there, and occasionally read my blog, thanks for that! I am going to write more. I took a great class last Fall-Methods in Human Computer Interaction. It was pretty intensive-I helped design a parking lot system-which is totally outside of my experience. That was incredibly useful. I think it helps to sometimes look at complex problems outside of your normal focus.

The class, because it focused on methods, introduced new methods each week. Some of them are so closely related, it makes one confused. But, my old favorites were there, although sometimes renamed. Bootstrap UX–who knew, a lot of folks call that Rapid Ethnography. There are some great articles out there: “Rapid Ethnography: Time Deepeining Strategies for HCI Field Research,” includes a nice case study that illustrates the main components of the method. I like this article because it confirms the importance of ethnographic research in the design process. The case study mentioned did replace open-ended interviews with a “condensed ethnographic interview,” which specifically covered specific project bottlenecks and benefits. The acceleration of this method gets at the heart of how I have stream-lined my approach. It states, “Time spent looking at the broad landscape is time that is not focused in the area of critical importance to a product design team,” and “While there may be nuggets of gold in the data, they are just too hard to mine.”  I love the concept of applying an ethnographic approach to usability problems.

I have to go mine some usability screen casts done last week now, but I’ll be back tomorrow!

Web Scale Discovery

I am fresh back from SLA 2012, Chicago, and while I have a lot of UX stuff to talk about, I am going to start with what I did first. On Monday, I was a part of a panel, Web-scale Discovery Implementation with the End User in Mind, with Rafal Kasprowski, Rice University; Debra Kolah, Rice University; Harry Kaplanian, EBSCO Publishing.

Harry provided an introductory historic overview of discovery, from the on-the-fly federated searching and accompanying catalog layers to the single index web-scale discovery, including some humor about the evolution of the card catalog. I talked about the method of choosing a tool, including a comparison grid, a ethnography study and the talented Rafal spoke about OpenURL vs proprietary linking, data and holdings migrations from ILS and resolver knowledge base, and product settings.

We had a lot of questions after the session–there is still great interest out in libraryland about discovery layers, and about half in the room of approximately 60 were actively thinking about purchasing a discovery layer in the near future.

The ethnography study I did for our discovery layer decision is unpublished, but I had read some of the following about some of the informants. (Note: the OneSearch referred to here is our previous discovery layer, Aquabrowser.)

Our informants fell into five major categories:

1)Three informants said they had never heard of OneSearch.

2)Five informants don’t use OneSearch, and do not understand what it searches. One informant stated, “I really don’t know what it covers, presumably it’s one place where you can search for everything, but if I’m looking for articles I go elsewhere, but if I’m looking for general information or books I would use OneSearch.”

3)Three informants have tried OneSearch but either want more control over the search, did not like something about the interface, or experienced failed searches when looking for known items. One informant simply stated, “It doesn’t seem to give me the answers that I want…I feel like I can’t control it to get the things I want.” Another informant stated, “I have used it only a couple of times, both of the times it was pretty useful…but it would be nice to pick which databases I would like to search.”

4) Only one informant uses OneSearch regularly and finds it to be useful. She uses it only with students, not for personal research, especially to search for items that she knows are not covered by the databases with which she is familiar. Emphasizing the wide net that discovery casts is useful to the undergraduate program.

5) One informant does not use OneSearch, but has created his own discovery-like tool using a Mac-based bibliographic program called Papers. The interface has allowed him, with assistance from Rafal and Debra , to identify frequently used Fondren resources, including Scopus, acquire special “keys’ for access, and then search these directly from this third-party interface, where is is identified as a Rice researcher. This user workaround will be interesting to monitor in the coming year, as some of the bibliographic tools, such as Papers and Mendeley, increase their ability for searching within the tool.

Our slideshow will be available soon on SlideShare. Additionally, here is the bibliography that informed my part of the talk:

Fagan, Jody Condit and Meris A. Mandernach, Carl S. Nelson, Jonathan R. Paulo, Grover Saunders. Usability Test Results for a Discovery Tool in an Academic Library. Information Technology and Libraries. March 2012.

Khoo, Michael, and Lily Rozaklis, Catherine Hall. A survey of the use of ethnographic methods in the study of libraries and library users. Library & Information Science Research. Volume 34, Issue 2. April 2012, pp. 82-91.

Promise Fulfilled? An EBSCO Discovery Service Usability Study (Aug 2011) - Illinois State University - Interesting comparisons with federated search usability studies and use of pre/post limiters and refinements. Journal of Web Librarianship,Vol. 5, No. 3. (July 2011), pp. 179-198.

Web Scale Discovery Services (2011) This work by Jason Vaughan is probably the first comprehensive work on Discovery systems so far available. There are chapters on WorldCat Local, Summon, Ebsco Discovery Service and Primo Central.

Wiki “Articles on Discovery” by Aaron Tay. https://sites.google.com/site/urd2comparison/articles-on-discovery

Ethnography Resources

Asher, Andrew and Susan Miller. So You Want to Do Anthropology in Your Library? or A Practical Guide to Ethnographic Research in Academic
Libraries. The ERIAL PROJECT. Accessed April 2, 2012 at http://erialproject.org/

Foster, Nancy and Susan Gibbons, eds. 2007. Studying Students: The Undergraduate Research Project at the University of Rochester. Chicago: Association of College and Research Libraries. http:docushare.lib.rochester.edu/docushare/dsweb/View/Collection-4436. Accessed July 7, 2012.

Small, Mario. 2009 “How many cases do I need?” On science and logic of case selection in field-based research.” Ethnography 10(1):5-38.

Bootstrap UX

I am a Texan, so the concept of bootstrapping is a way of life. Recently, I have been using it to describe the type of ethnography I am doing at my library. The very thorough Andrew Asher, ERIAL Lead Research Anthropologist, and Susan Miller, the Resident Anthropologist, ERIAL Chicago Libraries, describe in “So You Want to Do Anthropology in Your Library or A Practical Guide to Ethnographic Research in Academic Libraries?” a sample One-Year Timeline.

However, our library is considering the purchase of a discovery tool(DT) this summer, and we really wanted to include a small-scale ethnographic study in the process. So, what do you do?  You pull up your boots, and get to work! Even if you don’t have a year!

The team worked under very tight time constraints: the research instrument was approved on April 5, 2011, the interviews were scheduled from April 18-28, transcription and coding took place from May 2-13, and the final report was submitted to the Discovery Tools Working Group on May 18.

So that is basically, two weeks for interviews, and two weeks for transcription and coding! And, less than a week for the actual writing. Whew! But, I have to say, I think it was a great effort. All done in six weeks! We kind of lost one week, because I was gone from April 11-15th for the Texas Library Association conference. It was time well spent! Andrew Asher was there, and the Common Craft folks, and I saw Aaron Schmidt’s talk on user experience. Andrew’s insight into coding transcriptions was especially useful!

Our four person project team was able to do interviews with three faculty members, four members of a nondepartmental academic program, one postdoc, and five researchers. An additional person helped with transcription needs. I think four was the perfect number–I could not have done this with only two, and more than four would have gotten cumbersome in the coding process.

Yesterday, the main day of writing was a blast! At one point, there were three of us working real-time in GoogleDocs, with one of us out of state. We were able to use the chat function in Googledocs to ask questions, and clarify. Amazing! And fun! Ok, maybe I am a rare breed that thinks it is fun to work to deadline, but I was a school newspaper editor long ago, so it must be in my blood.

Throughout the whole process we kept the interview schedule in GoogleDocs, and then used Dropbox to keep the mp3′s while we were working on them. This cloud access assured easy access to project materials for all of the project members. At the final step, for the coded transcripts, and the final paper, I created a workspace in our university coursework system, so that it would be more secure, and longer lasting.

Now, there were a few things that I would do differently:

Important learning lessons from the DT study:

1) A good thing to capture would have been age/years of teaching/research. We will always ask this in future studies.

2) In a study we did last summer, we interviewed the subject bibliographer for biology, which provided some useful insight into the study. For the DT study, an informal meeting with the subject bibliographer was held, but it was not recorded. Once again, this yielded some interesting insights. It is highly recommended to include subject bibliographers in your study to inform the study.

3) On the DT study we did not specifically ask any question about what/how faculty taught or observed student research. It might be interesting to do this–to ask what professors see students doing in their research, what questions students have asked to clarify research assignments, and what the faculty member teaches the student about the research process.

4) Timing. Because we did the DT study so late in the semester, it was almost impossible to include students. Summer will be almost the same way. We will think of innovative ways that we could capture undergraduate students in the summer study. There are postdocs, and graduate students around luckily! I had just done some usability testing for our new website, so we were able to pull in some of that data into this study.

5) More time lets you roll in more techniques. It would have been great to do sessions capturing the users with screen capture software, or do a design project where users could design the perfect user interface for discovery.

But, all in all, the takeaway is: never let time constraints steer you away from incorporating user research in your decision making process! And we now have thirteen voices, in a four page report to guide our way.

Science + Ethnology

I am currently in the middle of a small-scale ethnographic study to find information that will inform a discovery tool purchasing decision committee. Additionally, I just held the first team meeting of a group of librarians that will do a study of how researchers find, manage, cite, and publish their research this summer.

So, I was very lucky when I happened to read a tweet written by an anthopology graduate student last week talking about Sharon Traweek. Gosh, I wish I would have discovered her 10 years ago! Beamtimes and lifetimes: the world of high energy physicists, written in 1988, captures the world of SLAC, and their labs, and really kind of reads like history of science now. Another book, Doing Science + Culture, edited by Roddey Reid and Sharon Traweek is a series of essays about how “cultural and interdisciplinary studies are changing the way we look at science and medicine.”

The discovery of these two books has me on a great literature search to discovery more ethnography of the discliplines that academic libraries support–these should be required reading in library school! Even if you are not doing a research project where you need to understand your subjects a bit more, it provides a powerful context for what your users experience in the lab and their world; this understanding can lead to better service design.