From 6.831Wiki

Contents 1 Group Members 2 Problem Statement 3 Target Users 4 Proposed Solution 4.1 Mental Web 4.2 Priority queue 4.3 Usage examples 4.4 Future work/Reach goal 5 GR1 Task Analysis 5.1 User Analysis 5.1.1 Types of Users 5.1.1.1 Experienced Researcher 5.1.1.1.1 Research Scientist 5.1.1.1.2 Professor 5.1.1.2 Inexperienced Researcher 5.1.1.2.1 Student 1 5.1.1.2.2 Student 2 5.1.1.3 Non Researcher 5.1.1.3.1 Person 1 5.1.2 Personas 5.1.2.1 Les the Researcher 5.1.2.2 Sam the Student 5.1.2.3 Joe 5.2 Task Analysis 5.2.1 Task 1: Login and Log Out 5.2.2 Task 2: Store a new node (article, research paper) 5.2.3 Task 3: Make a note (about a paper, relationship between 2 nodes) 5.2.4 Task 4: View and edit content in a node 5.2.5 Task 5: Browse and Search Nodes 5.2.6 Task 6: Manipulate Priority Queue 5.2.7 Task 7: Delete a Node 5.3 Domain Analysis 5.4 External Links 6 GR2: Design Sketches 6.1 Scenarios 6.1.1 Scenario 1 (Les the Researcher) 6.1.2 Scenario 2 (Sam the Student) 6.1.3 Scenario 3 (Joe the Plumber) 6.2 Preliminary Interface Designs 6.2.1 Design 1: Text-Based UI 6.2.1.1 Overview 6.2.1.2 Labeling(Tags, Domains, Priorities) & Adding Notes/Links to a Paper 6.2.1.3 Create New Item 6.2.1.4 Document Viewer 6.2.1.5 Search 6.2.2 Design 2: Graphical UI 6.2.2.1 Overview 6.2.2.2 Create New Item 6.2.2.3 Drag And Drop 6.2.2.4 Scroll Through References 6.2.2.5 Document Viewer 6.2.2.6 Search 6.2.2.7 View A Domain 6.2.2.8 Log In 6.3 Storyboards 6.3.1 Design 1: Text-Based UI 6.3.2 Design 2: Graphical UI 7 GR3: Paper Prototyping 7.1 Risk Assessment 7.1.1 Filtering by Tags 7.1.1.1 Novelty 7.1.1.2 Frequent Use 7.1.1.3 Error Risk 7.1.1.4 Complexity 7.1.1.5 Testing Scenario 7.1.2 Link Display and Organization 7.1.2.1 Novelty 7.1.2.2 Frequent Use 7.1.2.3 Error Risk 7.1.2.4 Complexity 7.1.2.5 Testing Scenario 7.1.3 Priority Queue 7.1.3.1 Novelty 7.1.3.2 Frequent Use 7.1.3.3 Error Risk 7.1.3.4 Complexity 7.1.3.5 Testing Scenario 7.2 Prototype Photos 7.2.1 Prototype 1 (Testing with Classmates) 7.2.1.1 Wizard Step1 7.2.1.2 Wizard Step2 7.2.1.3 Add Item 7.2.1.4 Wizard Step3 7.2.1.5 Add New Domain 7.2.1.6 New Domain Added 7.2.1.7 Network View 7.2.1.8 Search Cheeseburgers 7.2.1.9 Add Link 7.2.1.10 Priority Queue View 7.2.1.11 Changing Priority 7.2.1.12 Tags 7.2.2 Prototype 2 (Testing with Target Population Users Outside the Class) 7.2.2.1 Overall View (when Populated with Research Papers) 7.2.2.2 Wizard 7.2.2.3 Add New Paper 7.2.2.4 Add Tags 7.2.2.5 Search 7.2.2.6 Add Link 7.2.2.7 I'm Feelin' Productive 7.2.2.8 Important Items (starred) 7.2.2.9 Filter by Tags 7.2.2.10 Delete Paper 7.3 Briefing 7.4 Scenario Tasks 7.4.1 Prototype 1 7.4.1.1 Scenario 1 7.4.1.2 Scenario 2 7.4.1.3 Scenario 3 7.4.2 Prototype 2 7.4.2.1 Scenario 1 7.4.2.2 Scenario 2 7.4.2.3 Scenario 3 7.4.2.4 Scenario 4 7.5 Observations 7.5.1 Prototype 1 7.5.1.1 User 1 7.5.1.2 User 2 7.5.1.3 User 3 7.5.1.4 User 4 7.5.2 Prototype 2 (Baker user tests) 7.5.2.1 User 1 7.5.2.2 User 2 7.5.2.3 User 3 7.6 Prototype Iteration 7.7 Risk Resolution 7.7.1 Filter by Tags 7.7.2 Priority Queue 7.7.3 Link Organization 7.7.4 General Observations 8 GR4: Computer Prototyping 8.1 Platform 8.2 Accessing the Prototype 8.3 Depth/Breadth of Prototype 9 GR6: User Testing 9.1 Design 9.1.1 Look and Feel 9.1.1.1 Design Decision 1: New Graphic Design for Interface 9.1.1.2 Design Decision 2: New Login Screen 9.1.1.3 Design Decision 3: More functionality to make adding papers easier for the user 9.1.1.4 Design Decision 4: Scalable Add Tag Interface 9.1.1.5 Design Decision 5: Add help and documentation 9.1.1.6 Design Decision 6: Consistent design of menu bar pop-ups 9.1.2 Search 9.1.2.1 Design Decision 7: Search input textbox with double affordance 9.1.2.2 Design Decision 8: Search results displayed as list 9.1.2.3 Design Decision 9: Clicking on tag enables search by that tag 9.1.3 Graph 9.1.4 Details View 9.1.4.1 Design Decision 10: Side Panel or Front-and-Center 9.1.4.2 Design Decision 11: Grouping Controls with Display 9.1.4.3 Design Decision 12: Horizontal/Vertical Alignment of Tags Area 9.2 Implementation 9.2.1 Framework 9.2.2 Search 9.2.3 Graph 9.3 Evaluation 9.3.1 Scenarios 9.3.1.1 Scenario 1 9.3.1.2 Scenario 2 9.3.1.3 Scenario 3 9.3.1.4 Scenario 4 9.3.2 User Observations 9.3.2.1 User 1 9.3.2.2 User 2 9.3.2.3 User 3 9.3.2.4 Summary 9.4 Reflection

Group Members

• Yuetian Xu
• Zuoyu Tao
• Katherine Kuan
• Jenny Liu
• TA Mentor: Lydia Chilton

Problem Statement

There is currently no effective way to collect and organize all of a person’s research papers that they should read, and allow them to make notes that correspond to the papers. At any given time, grad students and research scientists will have many research papers to read. Usually they make note of this somewhere or save the document on their computer in a folder. Searching for that information or paper can be difficult (you don’t remember where you saved the information or the name of the file). Or the paper could just be inaccessible if it’s located on a different computer. As a result, people don’t have the chance to go back and actually read the article. They may have 50+ papers they should read but never get around to reading because of lack of organization. They also don’t have a clear order as to which paper they should read first.

This also applies to non-researchers such as everyday computer users who want to learn more about a certain topic on the Internet. They may collect a bunch of links to articles, but want to save them for reading later. Instead of saving them in a random text file on their computer, we’d like to integrate all these saved links into a system that allows for easy retrieval.

Target Users

We are targeting an audience of users who need to research or learn about a certain topic. These users may have difficulties remembering things or organizing things. Hence, when they have collected many articles and research papers on the topic, the system will help them keep track of papers they have read, how different ones relate to each other, where to find additional information. There will be 3 classes of users: Experienced researchers, inexperienced researchers, and non-researchers.

Proposed Solution

We propose a web application that organizes research papers and news articles. A web application would have the benefit of being accessible anywhere and thus more likely to be useful for this type of task. There will be two primary views/approaches.

Mental Web

This graph network view will allow users to create nodes which may be random thoughts/notes/tasks/research papers etc. They can use a rich set of tags to label both nodes and connections between nodes. These nodes may be as concrete as specific pieces of text or as abstract as asking a particular colleague about this domain of knowledge.

Priority queue

This view allows users to generate a priority queue for tasks/items to complete depending on the domain(s) of tags. This can be tied into the mental web representation using web analogies like Google Pagerank. Prioritization of specific areas can propagate the priority to related nodes.

Usage examples

• Case 1: Research Papers

A user may want to read a research paper, but might not have time to do so at the moment. The user can quickly skim through the abstract and create a node on MindDump of how this paper reminds him of other related papers. At this point, he can either explicitly assign priority to the task of reading this later on or let it remain implicit. Should he choose to elevate the importance of the particular field of research or a related paper, the weight can be propagated to this paper.

Later on, should the user have free time for reading papers, a priority queue can be generated for which papers to read first.

• Case 2: News Articles

A user may not deal with research papers but rather news articles. From looking at various news sources, he could amass a bunch of articles on similar topics. These articles could be grouped according to the topic, geographical reference, date written, news category, and etc... The user will be able to create links among articles that may reference each other or similar ideas and create a visual map of how the articles relate. They can also be assigned a priority level so the user can read them in a certain order from a priority queue.

Future work/Reach goal

Multi-user scenarios. Setting permissions for accessing specific parts of your mental web for friends and colleagues.

GR1 Task Analysis

User Analysis

Types of Users

Note: each subcategory described here is the result of a user interview.

Experienced Researcher

Research Scientist

• Uses email as ad hoc storage system due to ease of search
  • Email storage is ~500 emails
  • Email storage consist of two categories
    • Action items that are important but not urgent
    • Things that are already done but save saved as records.

• Gets ~50 interesting research papers to read annually
  • These are saved for vacation time/later
  • ~40% of these actually gets read
    • There needs to be a good prioritization scheme to maximize the efficiency and make sure the most relevant ones are read.
  • These may pertain to 4-5 projects going on simultaneously

Professor

• Email storage of 4000 (mostly unread)
  • Only clicks on the ones he thinks he wants to read

• Research method based on index cards
  • Easy to search
  • Grouped them by domain

Inexperienced Researcher

Student 1

• Uses email as task organizer
  • Some emails are to do's ~1/day
  • Email handling and prioritization
    • Actionable items
      • Urgent then reply within ½ hr
      • Non-urgent then star now and process later in the day
    • Non-actionable then file away

• Research
  • Must keep track of ~50 papers to read and maintain information about
    • For writing background of thesis later
  • Focused on just one project
  • Future papers to read/notes on papers read are written down in lab notebook
    • Drawback: not searchable
  • Important or cool ideas also jotted down in lab notebook
    • Attempts to label ideas etc. in margins but lacks consistency over time.

Student 2

• Email is disorganized
  • Go through 1x a week and read all
  • Drawback: often overlooks something important

• Research papers in a big folder
  • Drawback: lack of ability to maintain notes with papers
  • Flat structure

Non Researcher

Person 1

• Tasking tools
  • Remember primarily with own memory
  • Otherwise
    • Post-it notes in stack
    • Notebook
    • Very rarely emails things to him/herself
  • Drawback: Some information often forgotten
    • Always use forgotten password for some cases
    • Sometimes attach combo physically to lock
  • Drawback: need paper and pen

Personas

Les the Researcher

Les is a research scientist at a major research I university. He manages a couple of projects in the same domain, and he must read an extensive number of papers (50/year) to stay current. During his business trips/vacations he can only read about 20 of those papers. Our project should help him decide which papers are the most important for him, and which papers he can skim.

Les also receives approximately 15 emails/day related to his research. About 5 of them are immediate action items, and 10 are saved for later reference. He would like to be reminded of the emails he archived, but which are still important. For example, his buddy recently emailed him about Prof. Stanislaus Braun, who is an expert in a domain Les is interested in branching out into. Les archives the email. Later, Les has an opportunity to go to a conference in DC, where Prof. Braun will be attending. With our system, Les should be able to recall the contents of that email, and arrange a meeting with Prof. Braun.

Occasionally, Les gets some free time (perhaps while "vacationing" with his in-laws). He wants to update his mental representation of his domain, in order to get a better sense of where to go next. We should make it easier for him to do this, by allowing Les to sketch and update his mental map with our program.

Sam the Student

Sam is a typical college student at a small institution next to a river. Recently, Sam has been involved with Les on an exciting new project, but in a domain Sam has little knowledge about. With our software, Sam can organize the knowledge she learns from the project more efficiently using a graphical representation of her knowledge. When Sam has finished reading a paper, she can update that graphical representation of the domain. Les might even take a look at her representation, and correct it where necessary.

Sam can also use our software to recommend new papers to read. For example, Sam has recently finished a paper on the physics of wormholes. She wants to learn more by reading the cited papers. To figure out which paper she wants to read first, she uses our system to note that one of the cited papers was also recommended by one of her friends before. She decides to read that one first.

Later on, Sam thinks up of a great idea relating to wormholes. With our software, she can write down the idea and relate it to papers she has read. When a small crisis emerges later that month, Sam can now find that idea easily and apply it.

Joe

Joe is an average plumber suddenly catapulted to fame. He wishes to know more about the world, and our software can help him make sense of the news and punditry.

In Joe's case, he reads newspaper articles and watches TV shows, but the concepts are the same as mapping relationships between research papers and creating mental maps of research domains. However, Joe has some special needs to consider.

First, Joe is not an avid computer user. Therefore, he wants to take notes on paper. Our system might then suggest an indexing method for the notes that makes finding them much easier.

Second, Joe wants to focus more on creating a mental map, and less on finding new articles to read. Joe already gets too much information from talking heads on TV-- he just wants to organize it. So our recommendation system is not as relevant here.

Task Analysis

Task 1: Login and Log Out

Gain access to application and personal files

• Subtask: Type in username and password
  • Preconditions: Know username, password
  • Postconditions: Authenticate user and start application
  • Exceptions: Password incorrect
• Subtask: Forgot/retrieve password
  • Preconditions: Know username
  • Postconditions: Emailed password to user’s email
• Subtask: Create new user
  • Preconditions: Must have email address
  • Postconditions: Update database with new user and password, start application

Task 2: Store a new node (article, research paper)

• Subtask: Create a new node
  • Precondtion: Have the source (web link, file) location
  • Postcondition: A node with the information source
  • Frequency: High-- as often as the user wants to add another item to his/her collection of information items
• Subtask: Name the new node
  • Precondition: Have a source title name in mind
  • Postcondition: The new node is named
• Subtask: Save the new node
  • Precondition: A node with the information source
  • Postcondition: Node is added to the database

Task 3: Make a note (about a paper, relationship between 2 nodes)

• Subtask: Create new note for node
  • Precondition: A preexisting node to which the note is attached.
  • Postcondition: The node is attached to the node.

• Subtask: Create a new connection between two nodes
  • Precondition: Two different nodes exist in the network. They can be any distance from each other and can be connected or disjoint.
  • Postcondition: A connection is created between two nodes

• Subtask: Create a note about a connection
  • Precondition: A connection exists between two nodes.
  • Postcondition: The note is attached to the connection.

Task 4: View and edit content in a node

• Subtask: Easily view previously stored notes in a node
  • Precondition: User has found the node s/he wishes to view
  • Postcondition: The user has refreshed his/her memory of the note
  • Frequency: High--this is a primary goal of our system

• Subtask: Change the notes in the node
  • Precondition: User has something to add or correct
  • Postcondition: Expanded/Corrected note added to the node
  • Frequency: Moderate--users will probably spend more time adding more nodes, rather than changing the notes in existing nodes

• Subtask: Be able to view/edit multiple notes in a single node
  • Precondition: User has added multiple notes to the node
  • Frequency: Low--users may want to add more nodes to keep the organization more visible. However, if the note is long, it may make sense to divide it up into separate notes, but in a single node.

Task 5: Browse and Search Nodes

Quickly search through all nodes to find relevant one

• Subtask: Enter search query by label of node
  • Preconditions: User doesn’t know where to locate a node he remembers having
  • Postconditions: System searches database for node according to label
• Subtask: Display search results, allow mouse over to get preview of node before actually pulling it up in full-view
  • Preconditions: User entered search query
  • Postconditions: Display results in search window with preview options
• Subtask: Incremental search makes suggestion on what user may be looking for
  • Preconditions: Know what the node starts with
  • Postconditions: Presents user with options from existing nodes in system
• Subtask: Display list or index of all possible nodes in alphabetical order to allow browsing
  • Preconditions: User isn’t looking for anything in particular
  • Postconditions: Show all nodes in system

Task 6: Manipulate Priority Queue

• Subtask: Assign a priority to the node/Edit a priority by clicking on the node
  • Precondition: Know the priority of the node/Displays possible priority levels
  • Postcondition: Node is labeled with a priority
• Subtask: View priority queue(s) that the node is part of
  • Precondition: Click/hover desired node
  • Postcondition:　Highlights nodes of the priority queue(s) that the desired node is part of
• Subtask: View priority queue(s) with a certain label
  • Precondition: Have a label in mind
  • Postcondition: Priority queue(s) with the label are displayed by highlighted nodes
• Subtask: Drag and drop node(s) from one priority queue to another
  • Precondition: Highlighted nodes of priority queue(s)
  • Postcondition: Priority queue(s) are updated with exchanged node(s)

Task 7: Delete a Node

Remove nodes because their content is no longer needed

• Subtask: Erase a node from the network graph and priority queue
  • Preconditions: User doesn’t want the information in the node anymore
  • Postconditions: System removes the node from the graph and queue
  • Exceptions: No nodes to delete
• Subtask: Undo delete
  • Preconditions: User wants to retrieve something previously trashed
  • Postconditions: Restore recently deleted node back to graph and queue
• Subtask: Delete group of nodes with same label
  • Preconditions: Know existing label in database
  • Postconditions: All nodes with that label are deleted
• Subtask: Delete many nodes in graph at one time by graphically selecting the group
  • Preconditions: User doesn’t want a group of nodes anymore
  • Postconditions: Remove group from queue and network

Domain Analysis

Domain_Analysis_PPT

External Links

Important vs. Urgent Matrix link http://www.time-management-basics.com/images/top/Slide13.GIF

GR2: Design Sketches

Scenarios

Scenario 1 (Les the Researcher)

Les the researcher has a vacation coming up. Unfortunately, instead of going to some new and exotic location, he is stuck at home fixing a leaky roof and hosting visiting in-laws. He thought about it and decided that he would far prefer catching up on some paper reading instead.

He logs into MindDump. A few weeks ago, he had already assigned priority as well as labels to several papers stored in MindDump. Now, he gets to reap the rewards of the earlier work. He recently proposed a new project relating to machine learning. Thus, he opens the queue for machine learning papers. He was pleasantly surprised to find that the queue was almost exactly in the form that he wanted. He quickly makes a few minor corrections in rearranging a few items. Some other items he decides to move to other queues such as the computer vision queue as he decides those papers belong more over there.

He soon starts reading and immerses himself in the papers. He makes many notes on the papers that he's reading and draws various connections to the other related papers. As he is doing so, he realizes that one of the newly published papers had results that was superior to another paper in every way. He decides for the sake of keeping his MindDump small and efficient to just delete the old paper. With the click of a button, the old paper is deleted from the system. A clearly visible yet unobtrusive message reminds him that he can undo this easily had that click been accidental. Les is glad that the option is there since he does misclick every now and then. However, this time, he is confident in his choice having cleared out yet another mental cobweb.

Scenario 2 (Sam the Student)

Sam the student goes to her lab in the morning and finds links to two interesting papers in her inbox. These are from some of her lab mates and are said to be related to her research. She was just about to read them when her supervisor Les comes in.

"Sam, our sponsors are coming to visit tomorrow. I'll need you to put together a demo on the work that you've been doing so far. Also, look over these slides that we're presenting to them. I'll need them this afternoon." (exeunt Les)

Sam realized that as interesting as the papers are, they'll have to wait since these tasks she was just assigned by Les are a lot more urgent. However, due to the volume of email she gets all day, if she doesn't make a note of these papers, they'll quickly be buried at the bottom of her inbox.

Whatever shall she do? Just then, she remembered that she heard from her friend Bob about this new system MindDump. She went to the appropriate URL and created an account. She then logged into the system.

Once she was in the system, she was very impressed by the clean and functional UI. She was able to effortlessly add the papers in question to the system. She added the links to the papers and entered the title of the papers. From a quick glance at the abstracts, she was able to determine that one of the papers was more relevant than the other. Thus, she made that one have higher priority.

As she was doing this, she thought about an idea that linked these two papers together. Not wanting to forget this idea, she fumbles for a pen to jot this down in her lab notebook. Alas, a pen was nowhere in sight. Just as she was about to give up in exasperation, she realizes that in fact MindDump has support for this very task. With a few quick strokes, she was able to link up the two papers and make a note about the nature of the link.

Now that this is done, she can go back to working on that demo with her mind at ease that she won't forget about the papers or that cool idea she just had which would look great in her thesis.

Scenario 3 (Joe the Plumber)

Joe is sleepily watching his favorite web show, Saturday Morning Breakfast Pundits, when he suddenly realizes that the pundits on the show are starting to talk about his tax cut proposal! Hastily, he boots up MindDump (the best investment of time and money he's ever made) and logs in. Joe stores the web show link for future reference in MindDump. Next, he tags it as "About My Tax Cut Proposal". He notes that one of the pundits is the famous Crispy Matthews, and adds "Crispy Matthews" to the tags as well. Immediately, Joe can see similar news items stored in MindDump. One of them is the tax cut proposal he is writing, and another is a much ballyhooed debate between Crispy Matthews and Bill O'Really (which is now weakly linked to the episode of Saturday Morning Breakfast Pundits). Joe has never gotten a chance to watch it, but now that he is reminded, he can spend the rest of his morning watching talking heads.

Afterward, Joe is trying to find a news article about Bill O'Really and an alleged falafel incident. He remembers saving and annotation it on MindDump. However, with MindDump's powerful searching and browsing features, Joe can quickly find a list of items related to Bill O'Really and falafels. Joe can also follow the links between different items on MindDump, eventually ending up on the correct item. While rereading the article, he realizes that falafel stands deserve tax cuts too, and writes a small note on the article whilst embedding a link to his tax cut proposal.

Finally, after a long productive day of not plumbing, Joe logs out of MindDump.

Preliminary Interface Designs

Design 1: Text-Based UI

In the text-based UI, all items (documents/papers) are viewed in a simple list format. Each item has a Title, Domain name, Date Modified, Date Written, and Priority associated with it, where the list can be sorted chronologically or alphabetically according to those fields. Each item also has tags that can be associated with it (similar to the tags regarding the content in blog entries) and these are listed on the left sidebar of the list view.

By default, the first item in the list will be highlighted. At the top left of the screen there will be a corresponding box dedicated to the “Related Links/Papers” where the user can add references to other documents. The top right of the screen will have “Notes” where the user can add notes for himself about the paper.

The icon of a paper with a plus sign allows the user to add a new item by bringing up a dialog box with text fields for the title and URL of the document. There are also advanced settings in case the user also wants to add tags, a reference to another paper, or assign a priority to the paper.

When viewing the actual paper, we bring up the PDF version of the paper in Adobe Acrobat.

Overview

Labeling(Tags, Domains, Priorities) & Adding Notes/Links to a Paper

On the left column, the tagging feature, domain feature, and priority feature is available to the user. In the empty textbox below each feature's heading, the user can type in the desired tag name, domain name, and priority value. Once the user finishes typing the name, he/she can click on the little "+" button to add the name into the list of tags/domain/priority. To apply the label to the selected paper, the user just clicks on the checkbox to the left of the label, and the paper will appropriately be labeled with the name.

At the top of the page, the left section is for the user to add URL links related to the current paper. The "from section" shows papers that point to the current paper. On the top right, the user can click on the textbox and add notes about the current paper.

The papers can also be organized by the domain name, title of the paper, date modified, date published, etc. by clicking on the triangular arrows next to each of the headers for ascending or descending order.

Create New Item

There is an "add item button" to the left of the trashcan by the top above the "tags" feature. When clicked on the "add item button", the "Add New Item" window pops up. The user types the "title" into the textbox, and the "url" underneath. If the user already has "domain names" and "tags" in mind, then the user can click on the triangular shaped down arrow button to expand the tagging features. Once the user is done specifying the item, he/she may click on Ok. The undo feature is to click on "cancel" in case the user decides to longer add the item.

Document Viewer

When the user clicks on a listed paper, the appropriate application (adobe acrobat reader) will be used to open the file.

Search

At the bottom of the page, there is a search box, the user has the option to search through tags, domain names, or just the titles of the papers. The user types in a name, author, year, and the engine will return the list of results pertaining to the search.

Design 2: Graphical UI

In the graphical UI, we have a visual layout of all the domains on the home screen (i.e. for a computer researcher they would have domains such as AI, UI, Audio, Video, Speech, etc..) This will be a drag and drop interface, so new items can be created, dragged, and dropped into new domains. Depending on the action the user would like to perform, they can click on hyperlinks at the bottom of the home screen such as “Add New Item,” “Add New Domain,” or “Search.” Clicking on these links will bring up the appropriate dialog boxes.

To see what is inside a domain, the user can click on its icon and it’ll bring up a list view of the documents in that domain. We also have a graphical way to view the relationships among papers. The center holds a screenshot of a paper and above it is a horizontal, browse-able/scrollable list of papers that reference the center one. Below it is a scrollable list of papers that the center paper references.

To view any item, the user can just click the screenshot of the paper and will be brought to a full-screen viewer. The left sidebar will have the article information, tags, and notes. The right ¾ of the page will be the actual text of the article.

Overview

Create New Item

We wanted an easy and quick way to add a new paper/article, so we only require the user to type in the Title and URL. We might consider adding an advanced settings link that they can click to uncover more specific information such as tags, priority, references, and notes.

Drag And Drop

Once the item is uploaded, then there will be an icon of a paper in the center, which they can drag and drop into the different domains on the edges of the page. If the paper fits in multiple domains, the user can drag and drop it into multiple locations. There will be a "ding" sound that occurs when the user drags something into a domain as feedback that the action occurred successfully.

Scroll Through References

This interface allows users to view a paper at the center of the screen and the references to and from it. The top row is a horizontal scrollable list of screenshots of papers that link TO the central paper. If you use the mouse to hover over a screenshot of a paper, a small note will appear (if the user created one) about the relationship between the paper and the central one. The bottom row are the papers that link FROM the central paper. Again, relationships are defined in notes if you hover over the screenshot of the paper in the bottom row. The scrollable rows also show how many display results there are and what is actually displayed on the screen at that time (ex: Results 1-5 of 10). There is also additional information about the central paper on the sides of it such as tags and an option to add another paper as a reference. There is an incremental search on that text field so the system can suggest title names of papers already in the database.

Document Viewer

This allows the user to read the paper but also perform actions on the side bar such as classifying it with tags and notes. The user can also add references to other papers, search the document text, and assign it a priority.

Search

On the home screen, the user can click the hyperlink at the bottom of the page called "Search." Then a dialog box will pop up and allow the user to type in a keyword, domain, or tag. Then the search results will appear in list view similar to the next image below.

View A Domain

If the user wants to see what documents are inside a domain, he can just click one of the domains from the home screen. The upper left will contain an icon with the domain name. Then there will be a list of the documents with their title names, date information, and etc...

Log In

The user can type in his username and password to login. We will also probably add options to create a new account and a link if you forgot your password.

Storyboards

For our storyboard, we chose scenario 2 with Sam the Student who has two papers she wants to add, and paper 1 has priority over paper 2.

• Paper 1: Mitotic checkpoint genes in budding yeast and the dependence of mitosis on DNA replication

TA Weinert, GL Kiser, LH Hartwell - Genes & Development, 1994 - genesdev.cshlp.org Mitotic checkpoint genes in budding yeast and the dependence of mitosis on DNA replication and repair. ...

• Paper 2: Regulation of the cell cycle by calcium and calmodulin

- Full text - MIT Libraries KP Lu - Endocrine reviews, 1993 - Endocrine Soc ... Institution: Google Indexer | Sign In via User Name/Password ... The cell cycle was first thought to consist of mitosis and interphase as determined from ...

Design 1: Text-Based UI

1. First, Sam logs in with her username (SamtheStudent) and her password.
2. Add new domain "Biology"
3. Then she adds the two new papers
3a. Add new Item and enter title "Mitotic Checkpoint Genes" with URL www.mitoticgenes.com
3b. Same step is applied to "Regulation of the cell cycle" with URL www.cellcycle.com
4. Apply the domain name "Biology" to both of the papers.
5. Prioritize the papers so one paper has higher priority than the other

6. Create a note that links paper one and paper two together. The note says: "The cell cycle was first though to consist of mitosis."
7. Select paper 2, "Regulation of the cell cycle" Author: KP Lu- Endocrine reviews, 1993, Endocrine Society. ..."The cell cycle was first through to consist of mitosis and interphase as determined from...."
8. Apply a tag to paper 2, "genes" , "cell cycle"
9. Search for old paper, "Osmosis". Paper shows up, and she clicks on it.
10. Log out.

Design 2: Graphical UI

1. First, Sam logs in with her username (SamtheStudent) and her password.
2. Add new domain "Biology"
3. Then she adds the two new papers
3a. Add new Item and enter title "Mitotic Checkpoint Genes" with URL www.mitoticgenes.com
3b. Add new item with title "Regulation of the cell cycle with URL www.cellcycle.com
4. Drag and drop the paper to the biology domain

5. Prioritize the papers so one paper has higher priority than the other
6. Create a note that links paper one and paper two together. The note says: "The cell cycle was first though to consist of mitosis."
7. Click on paper 2, "Regulation of the cell cycle" Author: KP Lu- Endocrine reviews, 1993, Endocrine Society. ..."The cell cycle was first through to consist of mitosis and interphase as determined from...."
8. Apply a tag to paper 2, "genes" , "cell cycle"

9. Search for old paper, "Osmosis". Paper shows up, and she clicks on it.
10. Log out.

GR3: Paper Prototyping

Risk Assessment

Filtering by Tags

Novelty

Moderate

We intend to allow for filtering by multiple tags simultaneously. However, filtering results by multiple tags may be difficult to implement in an intuitive way. While certain other apps like Gmail also support search by tags, they do so via single tags.

Supporting multiple tags thus brings up the question of whether multiple tags represent an AND or an OR relationship.

Frequent Use

Very High

Due to the exploratory nature of the system, a search by tags is likely to be used as much as if not more than search by title.

Error Risk

Minimal.

Search only modifies the view, not the data behind it. Errors should be fairly easily recoverable.

Complexity

Mild

There is the precedence of label search in Gmail which shows all of them in a side panel. However, due to the nature of our application, we may have an order of magnitude or more tags. A drop down selection box akin to font selection in a word processor might work better.

Testing Scenario

Tested in scenario 4 of prototype 2.

Link Display and Organization

Novelty

Moderate

There are existing mind-mapping software as well as any number of UML or flow chart tools for displaying links. However, we may be dealing with again many more linkages than before. Furthermore, displaying the notes related to the linkage in a visually clear way may be difficult.

Frequent Use

Very High

This is our primary view for exploration. Showing linkages clearly is a central feature of the system and why the person might want to do that in the first place.

Error Risk

Minimal

Again, this is a display problem which likely wouldn't result in unrecoverable errors.

Complexity

High

While many software have links, many don't have notes relating to the links. Thus, making it clear that the link can be hovered over or clicked on to display a note might be unclear. Also, displaying many links on the screen at the same time may be difficult.

Testing Scenario

Due to the visual effects necessary, we do not believe this can be adequately prototyped with paper prototype. This will be saved for a low fidelity computer prototype.

Priority Queue

Novelty

Moderate

While many systems such as Netflix have some form of a queue, all of them tend to be a large monolithic one. Being able to filter the queue with tags would be extremely useful. You may only want to read about a certain domain etc.

Similarly, existing solutions would consist of a strong cardinal system for ordering. Netflix queue, for example, runs from 1 to however many you choose in a strict order. Instead, without reading a paper, it can be difficult to determine what the exact order should be. Perhaps we should have a rougher measure.

We chose as an initial prototype to just have the user star it if they want to come back to it later similar to the way Gmail does it.

Frequent Use

Very High

Another major mode of our system. As originally envisioned, this would be the dual of the web/graph representation.

Error Risk

Minimal

Again, this is a display problem which likely wouldn't result in unrecoverable errors.

Complexity

High

While other systems may have queues, using tags as a filter is novel. Users may not understand the connection.

Testing Scenario

Tested in scenario 3 of prototype 1

Prototype Photos

Prototype 1 (Testing with Classmates)

Wizard Step1

The first time the user starts the web application, a wizard will introduce the user to basic features to get the user started.

Wizard Step2

One of the first steps will be for the user to add a research paper into the database so the user will have something to work with.

Add Item

The "Add Item" dialog pops up. The user will enter a Title and URL for the paper.

Wizard Step3

The user will drag and drop the paper into a domain.

Add New Domain

When the user clicks on the "Add New Domain" button, an "Add New Domain" dialog pops up.

New Domain Added

The user added a couple more domains into the database.

Network View

This is the network view of our application. The papers (green post-it notes) on the top are papers that point to the current paper in the center. The papers (purple post-it notes) on the bottom are papers that are being pointed to by the current paper in the center. On the left, there is a text-box with a drop down for users to add a new paper to link to the current paper, "Add New Link". On the right are current tags (yellow post-it notes) of the current paper. The triangular arrows on both the left and right side allows the user to scroll left and right through the papers that are linked to the central paper. It is cut off in this picture, but on the top right and left corners, a display of "1-4 out of 20" papers are shown to let the user know his/her location as he/she is scrolling.

Search Cheeseburgers

The user can search for papers using the search bar at the top of the network view. The results will be displayed in a list view.

Add Link

The user can start typing the title of a paper into the text-box on the left-hand side of the paper to link the paper to the current paper displayed in the center. The user can also click on the triangular drop-down to get a list of papers that were recently viewed split and followed by papers sorted in alphabetical order.

Priority Queue View

The priority queue view presents the papers in list form on the left side of the screen. The papers can be sorted by "Priority", "Title", "Date Published", "Date Modified", etc. There is a preview of the selected paper shown on the right side of the screen. Below the list of papers, there is a legend of tag labels (which is shown in more detail under "Tags" section). Above the list of papers is a search box that allows the user to search through all the papers in the database.

Changing Priority

The user can change the priority of the paper, by clicking and dragging the paper to the desired location on the list. This will automatically change the priority of the paper. For example, if user wanted to move "Legend of Lol Cats" to Priority 1 from Priority 7, then the user clickc "Legend of Lol Cats", drags the paper on top of "Wizard of Oz Speech Experiment". Afterwards, the user will see "Legend of Lol Cats" at the top of the list with priority 1, and the rest of the papers shifted down by one priority (ie. Wizard of Oz Speech Experiment is now 2, Van Em De Boas Queues has priority 3, and so forth).

Tags

The user can apply tags to a paper by selecting the paper from the list, and then clicking on the checkboxes under the "Tags" list. After the user selects/checks the boxes of the desired tags, the user clicks on "Ok" and the tag is applied to the paper. The user can also select/check the boxes of the desired tags and click on "Search" which will then produce the results of papers with the desired tags in the list view. To create a new tag, the user would click on "New Tag" and a dialog will pop up, allowing the user to enter the new tag. The new tag will then appear under the "Tags" list.

Prototype 2 (Testing with Target Population Users Outside the Class)

Overall View (when Populated with Research Papers)

Instead of having multiple views, our second prototype only has one view. The left panel is a graph representation, the right panel is a list representation of all the papers that can be expanded horizontally to also include authors, date published, etc. Below the list view on the right panel, we have a legend of our tags, which the user can create tags, and apply tags to papers, or filter papers by tags.

In the left panel, we have a search box where the user can search for papers. To the right of the search box, we have a "I'm Feelin' Productive" button. Unlike the "I'm Feelin' Lucky" button, the user does not need to type anything into the search box. The functionality of the "I'm Feelin' Productive" button will be explained in more detail in the "I'm Feelin' Productive" section. The graphical representation shows papers linked to the central paper. The remaining icons will be explained in more detail in the sections below.

Wizard

Like our first prototype, a wizard guides the user through the basic functions to get the user started.

Add New Paper

The first step of the wizard asks the user to add a new paper. The user finds the "Add Paper" icon and clicks on it.


Here is a closer look at the icons related to the paper.


Once the user clicks on "Add Paper", a dialog pops up, prompting the user to type the title and URL of the paper.

Add Tags

The paper has been added. Now, the user wants to tag the paper, so the user clicks on the "Add Tag" icon.


A dialog pops up, prompting the user to enter a tag name. The user can click on the drop-down, which will provide a list of tag names for the user to choose, or the user can just type in a tag name.

The tags "Implementation" and "Machine Learning" have been tagged to the paper.


The tags "Implementation" and "Machine Learning" tags now appear under the list of Tags.

Search

The user can search for papers relevant to "Glyphs" and "UI Design"

The list on the right panel, shows "Glyphs" in the results.

The user clicks on the "Glyphs" paper in the list results and it pops up in the center with linked relevant papers surrounding it.

Add Link

The user wants to add a paper and link it to "Glyphs" so the user clicks on the "Add Link" icon.


A dialog pops up, prompting the user to enter the title of the paper he/she wishes to link to the "Glyphs" paper. The user clicks on the drop-down menu, a list of the recent papers will show up, followed by the papers sorted in alphabetic order. The user select "Legend of Lolcatz".


Once the user selects the title of the paper, the user can enter a description of link's relationship between the two papers.


"Legend of Lolcatz" is now one of the papers linked to "Glyphs", showing up in the graphical representation.

I'm Feelin' Productive

As mentioned previously, unlike Google's "I'm Feeling Lucky", the "I'm Feelin' Productive" does not require the user to type something into the "Search" box. Instead, the user just clicks on the "I'm Feelin' Productive" button and a random paper will show up.


The random paper shows up in the center of the graphical representation along with the papers linked to it. If there are starred papers in the list representation, the "I'm Feelin' Productive" will randomly pull a starred paper. (This starred feature will be explained in an example later on.)

Important Items (starred)

The user wants to mark "Legend of LolCatz" as important. The user clicks on the star icon next to the paper.


The paper becomes starred.

Filter by Tags

The user wants to read papers that have been tagged by "UI" and "Experiment", so the user clicks on the check boxes next to "UI" and "Experiment".


The result(s) of the filter ("SVM Decision Boundaries") shows up in both the list representation and the graphical representation.

Delete Paper

The user clicks on the "delete paper" button, which then deletes the "SVM Decision Boundaries" paper that was in both the graphical and list view.


The paper is deleted. The graphical view is blank and the paper "SVM Decision Boundaries" is removed from the list view.

Briefing

Thank you for participating in our usability test. Allow us to introduce you to MindDump! This is a unique web application that helps you organize your thoughts and papers when doing a research project. For example, if you wanted to research about bridges in the United States, you would surf the internet and find a whole bunch of articles on the topic. However, you wouldn't read all of them at the same time, just save the link in a list that you would get to later. This application allows you to maintain all those articles in a single workflow, tag them, organize them, make links among them, and add notes to the articles. Also, whenever you have time to actually go through the content, you can pull up MindDump and start reading the articles in order of importance. This also applies to organizing your research papers. For example, your professor sends you a bunch of research papers to read on Artificial Intelligence, but you can't read them right away. Store them away in MindDump for easy retrieval and reading later!

Scenario Tasks

Prototype 1

Scenario 1

You have just finished reading "Legend of LOL cats". You want to add the paper to MindDump, and add the tags "Implementation" and "Machine Learning" (robotic lolcats are a hot research topic).

Scenario 2

You want to find a paper related to glyphs and design patterns. Use MindDump's search feature to find such a paper, and then create a link between it and "Legend of LOL cats".

Scenario 3

You are extremely bored and want to actually read a research paper. Use MindDump's priority queue to find two different papers to read from the categories of UI and Speech. Then move a paper about lolcats to the front of the queue, since that is what you really want to look at.

Prototype 2

Scenario 1

You have just finished reading "Legend of LOL cats". You want to add the paper to MindDump, and then add the tags "Implementation" and "Machine Learning".

Scenario 2

You want to find a paper related to glyphs and design patterns. Use MindDump's search feature to find such a paper, and then create a link between it and "Legend of LOL cats".

Scenario 3

You are extremely motivated and want to actually read a research paper. Use MindDump's “I’m feeling productive” to find a paper to read. Then mark the lolcats paper as something important(starred) that you want to come back to later and read.

Scenario 4

Search for a paper that has tags “UI” and “Experiment”. First, pick a paper from the results list. Then, delete the paper.

Observations

Prototype 1

User 1

Using the Wizard:
Actions: Click to begin, Right click, Type in "lol catz", Click add new domain, Add new domain, Add tags implementation
Hesitation: drag and drop the paper to the domain Actions: Click on “networking”

“Ok um….”

“So I want to select the paper then add the tags, so I choose the paper I want” I click “these are the chapters?” “what are these?” “are these the titles of the book” “is the legend of lolcatz here?” “io click on that, and then I select, implementation and machine learning on the tags column”

“To check if it’s done, I click on another one, and go back to the paper to see if it is tagged w/ implementation and machine learning “is there an ok button that I click (to know that it is tagged)?”

Scenario 2

Clicked on domain view. What was the previous view? (did not notice the tabbing) Clicked on the Glyphs paper. “oh!” Hesitation, right-click glyphs—it remains highlighted—dragged glyphs into LOL Catz “I’m not sure how to create a link” Not clear what the colors were, “colors respond to tags” Skipped

Click on design patterns paper— “I’m really confused—I’m not sure how to create a link”

Clicked on “Domain”, Clicked on “Priority Queue”, “right click on paper”, “double clicking on each of the papers”

“Oh! Network View. Here we go! Alright new link, getting somewhere!” Clicked on “Add new link!”

Cursor is in the box right now, “clicked on arrow elements” “Now I’m going to look for lolcatz”  clicked on the “study of lolcatz “oh crap, how do I go back?”

“Alright, I finally now have these two papers”

“OH…this isn’t a selection box (referring to the incremental search)” “And then Add New Link! Done!”

“Ok, priority view”

Typed UI into the Domain. Click on , drag paper to the top of the priority queue.

Feedback: User didn’t know which bars were search boxes in the priority queue. 

User 2

Typed "Legend of LOlCatz". Clicked Add Item. User2 is confused after reading the wizard. User points at paper and asks "Is this supposed to be the paper"? “I don’t know what goes on next. I right click outside nothing happens. Right clicks on the wizard. Left click on the wizard nothing happens.

Clicked on Add new Domain. Drag and dropped paper into newly created mammals domain.

"Now I want to add a Tag. What’s a tag?"

Click on the search box. Clicked on Domain View. Clicked on Network View.

“I want to add a tag right? Add new link? Right click (on the paper)?”

“I want to find the paper of the legend of lolcatz”

“Domain view. No we just came from that, Priority queue.” (He has now arrived at the priority queue view.)

“I don’t’ know if this is clickable. Clicks on Legend of Lolcats. Clicked on new tag! Typed Implementation, and enter. I also check off the boxes, is there an OK somewhere “

“Find the paper related to glyphs…(reading off the directions) So I click on Glyphs. (Paper shows up)”

“Click on add link. Nothing happens. Click on the textbox, typed "legend of lolcatz". Then clicked on add new link.

Click on priority queue view. Search for UI under Tags. Only UI papers show up. Click on Speech. Click on the speech paper. Click on Legend of Lolcats and drag it up.”

Feedback: I think it’d be easy to click on the views, I think the problem is w/ the instructions. I had trouble finding the paths that weren’t visible.

Notes on User 2:

• Confused about how to put paper into domain—since there are no domains at first. Wizard is inadequate, since it does not cover how to make new domains.

• The existence of tags is hidden in the first (domain) view. Hard to discover different views. Unable to easily add tags in network view. User clicks “add new link” due to lack of “add tags” button in network view.

• Not easy to figure out how the links between papers work in network view. Papers on the top scrollable queue often mistaken as buttons.

• Drag and drop in priority queue relatively intuitive.

User 3

“I type in the Title and URL.“ Clicked on Add Item.

“So where are the domains?” Clicked on Add new domain.

Add new domain window pops up. Typed in mammals. “Is there any difference between tags and domains?” So I clicked on “mammal” domain.” Priority Queue view pops up. “ What’s the meaning of priority in this context? I’m a little confused about the priority and its meaning.” Clicks on paper and it pops up.

“So I drag this Implementation tag to this paper. Oh, the tag is not draggable? It's a filter instead?" (Only the Implementation tagged papers pop up.) “Oh, So how do I add a tag to the paper? So I click on the paper.” The paper pops up.

“I guess we already have these two tags (implementation & machine learning). I’m confused how to add these two tags. Oh ok, so there is no difference when I right-click on the papers. So I clicked on “add new tag”. Is that what I want? I already have a tag- so this button means it’ll add a new tag onto the paper? Not adding a new tag field to the paper? Or I can just drag the tag onto the paper?”

“What happens when I’ve clicked “add tag? I have to retype the tag name again?” (We added a drop down so he doesn’t need to retype the tag name.)

(Reading directions) “So I would like to find a paper that is related to glyphs and design patterns. I click on glyphs. Is this the only paper that I can find? I don’t know if the paper is related to design pattern, there is no filter for me to use?” (He can’t tell what domain the paper is).

“I will try to right click on this Glyph paper. What will happen?” The PDF pops up. “So there is no other button for me to use?”

“Hm, what’s the meaning of Network View?” clicks on network view. “Oh, add new link. What should I type? So, I typed the title of this paper—Legend of Lolcats, select Legend of Lolcats, and clicks “add link”.

Scenario 3.

“I go back to priority queue. Clicks on priority queue.” Reads over directions. “So I just choose the top two papers with the highest priority. Oh, (misread directions.) I now click on UI.” The UI papers pop up. “So I can choose these two papers. And points at Glyphs. Ok, so now I drag this Legend of Lolcats paper to the top.”

Feedback from the user: I think I was a little confused about network view. Maybe I can create a link just from the “Priority Queue” view. “It’s cool.” I think it’s a good idea to have multiple views, but I think every view should be able to do all the actions. I was curious how do you create a network by the papers. I’m not sure I would use the network view in the classroom setting. I’m not sure what you mean by domain and tags. Could be useful if I want to specify a paper in that domain in the priority queue view.

Notes on User 3:

• Tried to drag tags to paper in the priority queue view. Is it a good idea to make the tags drag and droppable?

• Confused about the colored labels in front of the boxes in priority queue.

• Mechanism for how to add tags unclear.

• User wanted to right click. We should probably support all functionality in context menus as well.

• User confused about paper with multiple tags.

• Need to make clearer the result of adding a new link. Probably dynamically update to and from lists. Should distinguish between the two?

• User was confused about the graph metaphor of network view.

 

User 4

Click to begin. Click. Click on Add new domain. Typed the URL, added the paper. (reading directions of drag and drop domain) “Where oh? Clicks on add new domain. Types mammals. Drags paper into mammals domain. Click on mammals domain. It goes to priority queue view.

Write mammals in the search box. Click on Legend of Lolcats. Paper pops out. “Oh, I see paper is already up. Click on checkbox, and then add tag. Oh, just click on add tag. “

Click on Glyphs. Glyphs paper pops up. “Can I go to Network view? And this paper just pops up?” So I click on the drop down menu, I type in Legend of Lolcats. And then I click “add link”.

“I would click on UI and Speech. Does it mean it would highlight things that both of them, or things that have one or the other? Then I will see what’s there (referring to results). And then oh, I am already in priority queue mode. And then I’ll drag Legend of Lolcats to the top.

Feedback: I didn’t really understand the domains in the beginning, but after we started talking about it, It was ok. If I clicked on this paper in one view, would it appear in the other view? And when we were originally in domain view, I didn’t expect to be brought to priority queue view. I don’t really understand what priority queue view was.

I wasn’t really sure what add tag was. The positioning of the add tag was misleading. I think there is a distinction between domain and tags. I don’t really know what the lines (referring to the links in the network view) mean. Could not tell that those were papers linking, things that you could click on?

Notes on User 4:

• Lack of existing domain confused user (also true for the others). Build in a default set? Set dependent upon user type IDed in wizard?

• Ambiguity as to whether multiple tag selection filtering is AND/OR operation.

• Tab names unintuitive.

• Hard to tell if changes saved while switching across tabs.

• Search tabs closer to search box to emphasize functionality.

• Need to make icons in network view look more like papers.

---

Prototype 2 (Baker user tests)

User 1

Notes on non-class user 1:

• User cannot easily find the add new paper button. Wizard needs to be modified for clarity.
• User suggests ability to add multiple tags at once.
• User wants a minimize button. Perhaps also an ability to see multiple papers at once.
• User slightly confused about how the links and papers are displayed—not sure what the “smaller” papers are.
• The graph display seems too busy to user. A simplified view with the smaller papers organized as a stack of papers.
• User has trouble with finding the “mark as important” button in the graphical view because it does not exist. Suggestion: allow functionality in graphical view.
• Search and filter are in two different places, user suggests placing the two together.

User feedback:

Organizes paper by very specific topics, like different genes. Suggests different research projects with different tags? Or using tags to organize research project. Priority queue is important. Set priority for papers by highlighting number of stars, a la Windows media player.

Overall suggestions:

• Need to make linked papers explicit.
• Lines would make links apparent?
• I’m feeling productive should be tagged w/ star. Be able to star from graphical view would be good.
• Advanced search option instead of filter box?
• Consistency across interface of having confirmation dialogs and not having confirmation dialogs
• Different tags for different research projects?
• Stack of papers sorted by importance? Being able to star them might not be enough.
• Drag and drop queue?

User 2

Notes on non-class user 2:

• User wants to add multiple tags at once.
• User wants ability to minimize active paper.
• Has used similar apps with the graph/links metaphor.
• User confused on how “I’m feeling productive” selects the papers.
• User confused on how to mark the papers as important.

User Feedback:

Delete paper button moved to the paper “toolbar” along with the add tags and add links buttons. User suggests that a Mac program called Pathway is similar to the graphical view.

Overall Suggestions:

• Users all seem to be used to using drop down for adding tags.
• In general, it is not clear to users that the results are what comes up upon search. Maybe the user attention should be drawn there.
• Add an option to delete paper in the task bar below paper as well.

User 3

Notes on non-class user 3:

• Wizard is confusing to user—add new paper vs. add paper
• User wants fancy animations for the add tags popup box—sliding out from the add tags button, or the task menu under the paper.
• Ability to add multiple tags
• User confused about how to use I’m feeling productive button—too similar to Google’s I’m feeling lucky button?

User Feedback:

Have the search results only appear when user do a search, otherwise, have the graphical view be fullscreen. Have a rating system with multiple stars.

Overall Suggestions:

• Add ability to do Ctrl-select to select multiple tags?
• Make it clear that in the graphical view, the objects surrounding the central paper are other papers
• Make it clear that top of the add links list is recently used/read papers.
• Different look and feel for I’m Feeling Productive compared with search.

Prototype Iteration

On the whole, Testing Day users were confused among the 3 different views (domain view, network view, and priority queue) and how only certain actions could be performed in particular views but not others, so we decided to get rid of them and combine the functionality into one. We also wanted to return back to our original idea of having a graphical view of the "network" of papers and how they related to each other (similar to a brainstorming map) instead of having rows on the top and bottom of the screen with papers related to the center one. The metaphor would be easier to understand. Also, we modified our priority queue idea so it would be easier to understand for real users who weren't familiar with this computer science concept and made it more usable.

• Removed concept of domains (too similar to tags) and dragging/dropping papers into domains
• Bidirectional relationships among papers (links will mostly be used for relating papers with similar content, so typically bidirectional)
• Modified wizard to include newer simpler version of adding an item to MindDump
• New task bar below the central paper with Add Tag, Add Note, and Add Link functionality
• Visual graph network of a central paper with links to related papers completely surrounding it by 360 degrees
• "I'm Feeling Productive" button to pull from the front of the priority queue
• Slidable list view on the right side of the screen (displays alphabetical list of all papers in system or search results)
• Manual priority labeling system by "starring" papers (similar to Gmail)
• Manage tags interface

Risk Resolution

There were actually several things which we didn't consider particularly risky but came up during user testing. Most of these we addressed in the prototype iteration section above. The rest are included in the General Observations subsection below.

Filter by Tags

In our first prototype, the fact that boxes next to the tags were checkboxes seemed unclear. This can be likely addressed via computer prototyping where this becomes more clear. This might be worth going back to.

In our second prototype, we made the system behave as if the multiple selection was an AND operation. This seemed fairly intuitive but that might just be due to what we made our task.

Perhaps two links above the results saying "filter within results" and "add results from another filter" would work well.

Another observation we made during the second prototype was that we should move the search bar closer to the tags to emphasize that they are related in function and can work in tandem.

Priority Queue

From our paper prototyping, we found that rearranging the queue with drag and drop was extremely intuitive for the users, perhaps due to experience with things like Netflix.

We also found that having just one star is insufficient granularity for the user.

The best compromise in terms of control and simplicity is likely somewhere between 3 and 5 stars, similar to the way rating systems for music and movies work. The user may have a rough sense of the priority of the paper but most likely would not know exactly how important it is. We may instead want to rely on tag filtering to differentiate papers.

Link Organization

Not tested in paper prototypes as stated above

General Observations

In both prototyping sessions, we need to be able to allow the user to perform the same actions multiple ways. While this is partly alleviated by switching to a single screen interface, it is still of paramount importance.

Users want good separation between UI elements which offer different functionalities and count on certain accordances. During debriefing, they stated that search box and tags being close to each other and/or same color would aid greatly in knowing they offer similar functionality. They also stated that another button near a text box implied that one needed to type text into it to have it do something. This is a problem with the "I'm Feeling Productive" button that we introduced in prototype 2.

GR4: Computer Prototyping

Platform

Our system requires Flash Player 10 and Windows, using Internet Explorer. Our recommended resolution is 1024 x 768.

Accessing the Prototype

Click here to download the Zip file. Extract and open the HTML file. Note: the SWF file would only work if you have the debug version of the Flash Player here This should launch the MindDump application in a browser window. Maximize the window for best viewing experience. You can create a new user and password or just hit the Login button automatically to enter the application.

Depth/Breadth of Prototype

• Interactive graphical view of papers implemented in depth
  • Clicking on peripheral papers to swap into the center position
  • Mousing over peripheral papers to show full title and pictures
• Partial depth of incremental search
• Partial depth of star rating system
• Partial depth of tag filtering
• Partial depth of central paper options (Add Link, Add Note, Add Tag)
• Login page, Menu bar items

GR6: User Testing

Design

Look and Feel

Design Decision 1: New Graphic Design for Interface

Again, after user testing, we decided to change the whole look-and-feel of the site. We received feedback that it was "too visually complex" with too many scrollbars everywhere because it wasn't resizing properly on people's screens. Instead of a Flex HDividedBox (which would introduce scrollbars), we decided to just have the same header and title go across the entire screen instead of having the search bar take up the entire left side of the screen. Instead of having the search bar be collapsable with a movable divider, we left the search panel static on the page on the left side because most users left it open during paper prototyping.

This made the components of the application appear more seamlessly connected. We also introduced a new color scheme. We received user testing comments that the colors we used before (cyan, magenta) were too saturated and too visually stimulating. By using what we learned in class about sticking to only a couple colors for a color scheme, we decided to consistently use the same shades of light blue, white, light gray, and dark gray throughout the new version of MindDump. The font was also standardized across the app to be Verdana 12.

Another important decision was to keep everything on one page (graph, search, and menubar). From paper prototyping where we had 3 separate tabbed views (graph view, network view, and priority queue), we realized that users wouldn't be able to distinguish among them. It would be a hassle to switch between them so we decided to just have everything manageable from one page as seen above.

Design Decision 2: New Login Screen

After user testing, we decided to change the login screen. The computer prototype version didn't use all of the screen real estate (there was lots of white space previously) and the look-and-feel also didn't match the MindDump application. Thus we decided to make it have the same header as the application as you see above. We also added a short 3-point summary (create, organize, view) on the purpose of MindDump so users could easily remember what the application was intended to help them with.

We also added helpful error messages to the login page to for help on how to fix the error (i.e. wrong username/password, created a user with a name that already exist).

We also put "Create new user" and "Existing User Login" on the same page so that it would be efficient for both types of users and you could easily switch between the two (another caveat that was mentioned in our user testing feedback). In addition, people mentioned that the boxes were misaligned because of different screen resolutions. We fixed this by using the Flex Form component and having everything line up automatically.

Design Decision 3: More functionality to make adding papers easier for the user

Instead of having a blank text input box for the user to specify the location of the paper, we decided to take the advice of one of our testers by having a "Browse file" button so that users could search for a document on the file system. We also added better labels so that users would know that they could add a document from their file system or a web URL, but not both.

Design Decision 4: Scalable Add Tag Interface

Our tag interface started off as something similar to colorful filters and tags like in Gmail. However, after many iterations from paper prototyping and user testing, we changed the tags from buttons, to checkboxes, and then finally to a list component. Now you can view the list of existing tags for the paper and the list of existing tags for the whole system side by side and use the "<" arrow to move tags back and forth. This was the most scalable solution and allowed for single or multiple selection as well as easy sorting. Another related decision was the decision not to have "folders" and "tags" to organize related papers, but rather just use tags because their functionality was redundant almost. This was decided after paper prototyping.

Design Decision 5: Add help and documentation

Besides being one of Nielsen's heuristics, users were sometimes confused on how to perform certain tasks during paper prototyping, we added a help and documentation link in the menu bar. This described all the features of the application.

Design Decision 6: Consistent design of menu bar pop-ups

We made many cosmetic changes that would maintain consistency in the UI such as making pop-ups look similar (same layout of buttons and forms, consistent terminology of "done" versus "cancel", or same location of "OK" and "Cancel" button like in the Mac vs. PC). This was brought up by people in user testing because our manage tags (for the entire system) and add tag (for just that paper) looked different, so we made this consistent. Other things we added were: obfuscating the password on login, hide menu bar options when haven't logged in yet, centering pop-ups when they pop up, being able to hover over the menu bar and/or image to select that menu bar item.

Search

Design Decision 7: Search input textbox with double affordance

On the left of the above image, the Search Panel is displayed. Our goal was to have a minimalist design for the search panel. Thus, we use the double affordance of having default text "Type paper name or tag here." displayed in the textbox. By using double affordance, we save space and also remove the necessity of labeling the textbox with a "Search" and/or using a "Search" button. In our computer prototype, we originally had a "Search" button to the right of the textbox. However, our search has the capability of incremental search, displaying results as the user types his/her query into the textbox. This removed the necessity of a "Search" button. This was a result of a recommendation made to us in user testing.

Design Decision 8: Search results displayed as list

We wanted the presentation of the results to be consistent with other search engine results to increase learnability. Thus, our list presentation is consistent with internet search engine results, ie. Google.

Originally, we presented our search results in a datagrid(table). Each row would represent a paper, each column would have the Title, Author, Tags, and Rating of the paper. However, from our computer prototype which used the datagrid organize the results, we realized that using a table cramped all the information of the paper to fitting on one line. To view the title of a paper completely, the user would need to readjust the column width accordingly. To reduce the hassle, we decided to present the search results as a list as seen in the screenshot. By presenting the results as a list, the information is plainly visible, the user does not need to readjust columns to see the full title, nor readjust the columns to see all the tags, etc.

Similar to how Google presents the title of the webpage as the first line, we present the title of the paper on the first line and bold the title of the paper, and then include the tags of the paper as hyperlinks under the title, and finally include the star ratings of the paper on the bottom right.

Design Decision 9: Clicking on tag enables search by that tag

Similar to blogs, we realized that it would be convenient for the users to be able to click on the tags that are associated with papers that come up in the search results. Therefore, if a word reminds us of another keyword, we can search on that word too with a single click instead of typing the whole word out in the search box.

Graph

The graph was designed to be pleasant and easy to navigate. For the most part, we focused on making the graph intuitive. To do this, we made many actions achievable only through left clicking. No context menus were used, and things that seemed clickable were (other nodes representing related papers, the central paper). This was borne out during user testing, when most users had no difficulty locating papers that were connected to the original paper (of course, unlinked papers are impossible to find in the graph view).

The springiness of the graph contributed to the satisfaction of the users. We found that animations were lacking in our prototype, so we put in feedback when switching to the different papers. Now, when the user switched papers, the graph will give a little "bounce" and the paper will then spring into view. Unfortunately, because of the algorithm used, we had the problem that the nodes representing papers would not always stop in the exact same place when the users navigated to different papers. However, since the papers were clearly labeled, we consider this inconsistency to be merely cosmetic.

Another factor in the design of the graph view was simplicity. If we always displayed the paper screenshot, we would quickly run out of room in the graph view. To remedy this, we only displayed the screenshot for the central paper, and displayed screenshots of the related papers as tooltips, activated when the user hovered over the appropriate node.

We also wanted user control and freedom in our design. We gave the user the ability to drag and drop the entire graph structure (so it would fit better on the screen), and also to drag and drop individual nodes in case they were obscuring important information, or just for reasons of asthetics. We also added in a slider that allowed the user to control how far away the related papers were, relative to the central paper. (Below is an example of having a very long link distance). This allowed lucky users with a large monitor to be able to utilize more of their screenspace.

We discussed different options on how the user would actually read the article (such as PDF viewer, or a new interface), but we ended up launching it in a web browser, which makes the most sense. It displays online new articles, so that users can browse that web site for other articles that might not be in MindDump. Also it can display PDFs or Word doc files from a person's computer file system. Below is an example of an article launched in the browser window next to it.

Details View

Purpose of the details view is to allow a way to easily visualize the metadata (tags, notes, and links) associated with the currently selected paper.

Design Decision 10: Side Panel or Front-and-Center

In earlier iterations, the functionality provided by the details view was integrated into the graph with a panel of buttons underneath the paper to perform maintenance on tags, notes, and links. User feedback reported lack of ability to see these data fields.

One potential solution adds additional display elements around the selected paper. While this would give a clear grouping of the metadata elements with the central paper, the additional screen real-estate required would either shrink down the central paper more or take more space from the other parts of the graph. Furthermore, the padding that this would create makes the links to the central paper less clear.

The final design made the panel occupy the same space as the search panel, using a tab analogy. This had simplicity benefits in having just a single panel to the left as well as being economical with screen space. While the grouping with the central paper wasn't quite as strong, the information scent provided was more than strong enough as validated by the user tests.

Design Decision 11: Grouping Controls with Display

Another important aspect was whether or not to group the modification controls with the display controls. This is related to the previous decision. One approach is to move control buttons originally underneath the paper to a bar on the new details panel. The display areas will be distinct and perhaps underneath that.

The final design chose to incorporate both the controls and display together as it provided better Gestalt grouping this way. Also, the other design would have ran into a potentially awkward mapping problem where the controls for adding tags, notes, and links are placed horizontally while their corresponding viewing areas are arranged vertically. This would have resulted in the same type of mapping issues that a stove top has with displaying front/back burners as discussed in class.

Design Decision 12: Horizontal/Vertical Alignment of Tags Area

With notes and links, it's fairly clear that the text associated for display will be fairly substantial in length and would thus require entire lines to themselves. Thus, it makes sense to arrange the notes and links in their areas vertically, one per row.

The question now remains whether to do the same thing with tags. Do we, in the name of internal consistency, apply the same design to tags to have one per row? Do we instead list them consecutively instead in order to save space?

In this case, due to the vastly larger number of tags than either notes or links (part of task/domain analysis) as well as the expected shortness of the tags, we decided to sacrifice a little bit of internal consistency in this case. To place each tag on its own line would result in lots of empty whitespace on each line as well as a tremendous amount of vertical scrolling which would be inefficient use of screen space and yield bad visibility.

Furthermore, designing the system this way actually creates more external consistency with tag display on many websites such as Twitter and Stack Overflow

Implementation

Framework

MindDump was implemented as an Adobe AIR desktop application. It originally started off as a Flex Web application, but we switched over because of the need to access the user's local file system for browsing/uploading their files into MindDump and persisting user changes to the system. There was a central MindDump class which acted as the "controller" in MVC and took in user input and coordinated the appropriate changes to the model and graphical view. The menu bar across the top was intended to be static so that was built into the MindDump class. Upon login, the MindDump class would add the OptionsWindow component (search/paper details panel on the left) and the GraphWindow component (showing the graph of papers) to the window. Upon logout, the MindDump class would swap out and remove the OptionsWindow and GraphWindow components and would add in the LoginWindow component to show the login screen.

Clicking on the items in the menu bar would bring up pop-ups. Each of the pop-up windows were implemented as individual Flex components to easily abstract away the details of each window. They would fire events such as "Add Tag" or "Add Paper" and the MindDump class would listen for changes and update the model. These were custom defined events because they needed to pass information with them such as the a string for the tag name or a paper object for the paper.

Search

In our model, we created Paper objects that had Title (String), Tags (Vector of Strings), Notes, Links, Rating(Star Component). The search algorithm is very primitive. It iterates over all the titles and tags of each paper and sees which Paper objects matches the query. All the Paper objects that match are then pushed onto an array. The array is then read, and the information paper title, paper tags, and paper rating are extracted and appended as children onto the search result panel.

The star component is a custom component that we created ourselves. Toggling the image changes the star from gray to yellow, and vice versa. Depending on the position of the clicked star, the stars to the left of its position will all be toggled to reflect the proper rating.

Graph

The graph implementation was done by extending an Adobe extra component called Springgraph. This provided the animation functionality as desired.

While the initial concept was for a full web/graph representation, we ended up restricting visible graph depth to be 2 (displaying only direct neighbors of the central one). This is due to both limitations on screen size as well as animation time.

With nodes that have large degrees, the screen will often be inadequate for displaying all the neighbors of neighbors gets messy. Thus, for the sake of compatibility, we restricted the display. Perhaps it would make sense to enable the user to customize the depth should they have a large screen (say 24"+).

The animation time also becomes an issue with more graph depth. Due to the physics type spring analogy, the link lengths and positions will take some time to settle. For an aesthetically pleasing animation, the dampening effect must not be too strong. Without strong dampening, having bigger graph depth means it takes a long time for the graph to settle.

Other interesting implementation issues came up due to poor coding of the Springgraph component. It appeared the Flex component was merely ported from a Java implementation, and several bugs arose because of it, such as the graph being off-center. This was due to incorrect initialization of the "x" and "y" coordinates. Also, because of the need to work around its Java heritage, SpringGraph had a bizzare way of rendering components, with the end result that there was pretty much no modularity or encapsulation possible. For future work, we do not recommend using SpringGraph unless one has a high tolerance for pain.

Evaluation

We briefed users by explaining an overview of MindDump. We told them how it was an application designed to help them organize their research papers and news articles, so that they wouldn't have a bunch of random links and PDFs of things to read cluttering their email inbox. They simply can enter the paper into the MindDump system, and organize everything there.

Scenarios

Scenario 1

Your professor emails you a paper that they think you should read. Add this paper (located at http://www.time.com/time/business/article/0,8599,1895740,00.html?cnn=yes) to MindDump. Enter it into the system so that the title is "Facebook Hacking Problem" and the picture corresponding to the paper is "facebook.png." Then add the tags "facebook" and "hacking" to the paper. Then add a link to another existing paper called "Facebook losing its glow?."

Scenario 2

You attended a talk about the iPhone and you remembered you had a bunch of articles that you never got around to reading, so you pull it up, in the talk you were especially interested in iphone applications, so then you filter by applications. So first, search for "iPhone" and then filter by tag "application."

Scenario 3

Change the rating of a paper to be 5 stars. If you hit recommend me a paper, it should pull up that paper. Add a note to it. Edit that note.

Scenario 4

Open up "At Wireless 2009: It's All About the Apps" paper. Browse the graph window. Click on a linked paper. Launch the paper in a browser window.

User Observations

User 1

User 1 was a Macintosh user, so she did not right click often, unlike the later users who were slightly confused when right click didn't seem to do anything.

She had problems figuring out how to add web articles to MindDump, since there was no tutorial on how to add screenshots. She thought that if the screenshot didn't display correctly, the web article would not have been added to MindDump.

In the second scenario, she was able to figure out how to add tags and links to existing articles. It seemed intuitive to her.

When asked to search for particular papers using the search interface, she did not know how to filter the articles by their tags, perhaps due to confusingly worded instruction. Nevertheless, she figured it out in the end, but expressed disappointment that there was no way to filter by multiple tags at once.

One common complaint that all the users had was that the box where we put notes related to a paper looked too much like a text input box. She kept clicking there, expecting to be able to type in the notes directly, and only afterwards did she notice the "add note" button. She also commented that although when adding notes, the button was "Add", when adding tags, the button was "Edit", which could be a minor consistency issue.

Finally, she suggested a way to zoom in to the central paper being displayed, (presumably as a recognition aid) since many web articles looked the same with a low resolution preview.

User 2

User was temporarily confused by the necessity of adding a screenshot of the page in question by manually taking the screenshot and browsing for it on local file system. This issue is expected to be addressed when we receive an appropriate license and thus can use a third party software package to handle this automatically.

Some confusion over location of the add tags dialog. Possible fix is to increase information scent or remove the global manage tags interface.

The dual combo box setup for add tags was considered intuitive by this user.

User initially confused about existence of a paper in the system. This will likely not be an issue if the user added the original set himself. Increased familiarity with the system will also allow the user to confirm the existence of papers in the system via use of the search panel.

Filter by tags is confusing for the user. It’s unclear that the hyperlinked text represent tags. If the user had more experience with websites like Twitter and Flickr that use tags extensively, we would have the affordance built in that these small pieces of hyperlinked text are in fact tags. Another aspect is that we switch the system from a web app to a desktop AIR app in anticipation of getting the automated screenshot application to work. On the desktop, there aren't really the same set of expectations regarding tagging and hyperlinks. This is where perhaps many affordance break down.

Add note is somewhat difficult to find. A bit of delay on the part of the user in finding that it’s under details panel. It's interesting to note that after a small delay, the details panel was in fact the first place that the user clicked. However, other ways of adding information scent will still likely proven useful. Perhaps a byline for the details panel that talks about editing and viewing tags, links, and notes. Search panel could have a similar one talking about filtering and sorting.

What we meant by graph window is quite unclear. Explicit label or intro demo would make things a lot clearer. Note that this will probably not be a problem in practice as the user doesn't really need to know what the canonical name for this widget is so long as said widget is useful. Perhaps we are being too explicit with the directions for the scenarios? Perhaps higher, directive level instructions would encourage exploration and get around this.

It's unclear that a paper launched in a browser window. The user does not expect results to essentially occur outside the app in the background. This can be fixed by making this app a web app again which would put this in the browser as well, making it easier to notice the change (active new tab would steal focus). Another possibility is to explicitly have the new browser window steal focus.

User 3

User completes scenario 1 without any difficulty. User finds “edit tag” immediately and knows to apply the tag to the paper by clicking on the “left arrow” to add tag to the paper. The user also knows to type “hacking” into the textbox and clicking on the “left arrow” to apply the tag to the paper. User finds “add link” successfully and adds link without difficulty.

Usability Problem 1: Learnability of Tags as hyperlinks
User semirecognizes tags as hyperlinks. The user explored for about 10 seconds by clicking around before realizing that the tags were represented as hyperlinks under the title of the paper. To increase the learnability of the tags, we could introduce consistency by adding color to the tags as seen in other applications such as the labels in Gmail, the tags in RememberTheMilk.

Usability Problem 2: Learnability of Opening Papers in Graph View
The user tries to open the paper in graph view by clicking on “add paper” after searching for “Wireless 2009” paper under the scenario “Open up wireless 2009”. The user expected that searching for the paper in the search panel will automatically bring up the paper in the graph view. After clicking around and exploring for about 10 seconds did the user realize that double-clicking on the paper “Wireless 2009” pulls up the paper in the graph view.

Usability Problem 3: Lack of Feedback that the Article is Loaded in the Web Browser
User does not recognize that the paper was loaded into the browser after double-clicking on the paper in the graph view. This problem can be fixed by bringing the web browser into focus instead of the desktop application, thus providing feedback to the user.

Suggestions from the user: “If you have a lot of paper and tags don’t cover them all, list all the papers in a folder way. A way for all the papers to be listed would be nice. I’m guessing it would be less used since it would be annoying to go through all of them."

Summary

Since these were new users, we probably should have had them start with a wizard of some sort to give them directions on how to use the interface and to direct them on exactly what to click. We got rid of the wizard because of the limitations on user freedom, but after user testing I think it might be good to add it back in. Even though users got stuck, afterwards they thought that the way our application worked made sense, so the usability problems we experienced might be focused more on the initial curve than on efficiency of the interface. We also learned that we should probably allow users to do things in multiple ways so that they could click around and explore and eventually stumble upon one of the multiple ways to accomplish a task.

Reflection

In general, this was a great learning experience in how to create a UI through an iterative design process. It gave us a new perspective on the design of software, websites, etc. This experience and perspective should prove very useful going forward. It also made us more cognizant of the user and how important it was to design for them, not to design it the way we think they want it.

We believe that our risk assessment was pretty accurate. The parts which we deemed to be risky did turn out risky such as the priority queue. It was pretty interesting to see how our initial concept of a priority queue changed into an "I'm feeling lucky button" into a "Recommend me a paper" button, and we still think that this feature needs much more improvement. I think these changes came about because we realized that we couldn't do everything for our UI, but instead focus and prioritize on what our users' main purpose would be.

Another observation was that the prototypes did not approximate the final form very well. Things such as a drop down menu that seemed to work in paper protoyping turned out to be a scalability problem in our UI. In any case, these issues might be due to the fact that we took on the design of a fairly ambitious system. The system of our scope was something that we continually had to define throughout the semester. It could have been very broad to include something like Nota Benne for reading papers and etc.. Our system would definitely have benefited from another cycle or two of iterative design. For example, doing user testing and feedback, even on an informal level, outside of the GR's might have been very useful.

In terms of implementation, we were surprised by how long many of the relatively small pieces (designing dialogs etc.) would end up taking. On that note, we're glad that we had to do so much planning ahead of time because we realized how much time it took to rehaul the entire UI (like we did from GR4 to GR5). As Prof. Miller mentioned in class, usability is the tip of the iceberg, and the backend is the majority of the iceberg. In our iterative design process, in the final phase for GR5, our system would have benefited if we balanced the time spent on the backend with the time spent on the front end. Our final product changed drastically from the computer prototype. It may have benefited us if we conducted another round of user testing on a shallow prototype of our final product before implementing it fully.

Overall, we learned that user interface design was very difficult. What made clear intuitive sense for us could be very confusing for someone else. In that respect, user testing at the end was very eye-opening (even after weeks of iterations, testing, and redesigning when we thought we got the design somewhat right). There really is no end to user interface design. In our group discussions where we made decisions, we realized that for every way there was to design something, there would always be counterarguments against it. In the end, it was about balancing trade-offs and trying to understand what would be most helpful for our users.