Teaching abstract concepts is notoriously difficult, especially when we lack concrete metaphors that map to those abstractions. Combinatorix offers a novel approach that combines tangible objects with an interactive tabletop to help students explore, solve and understand probability problems. Students rearrange physical tokens to see the effects of various constraints on the problem space; a second screen displays the associated changes in an abstract representation, e.g., a probability tree. Using participatory design, college students in a combinatorics class helped iteratively refine the Combinatorix prototype, which was then tested successfully with five students. Combinatorix serves as an initial proof-of-concept that demonstrates how tangible tabletop interfaces that map tangible objects to abstract concepts can improve problem-solving skills.

Introduction

Many decisions benefit from understanding probability, e.g., when a patient must interpret the meaning of a medical test result or when a politician must weigh the costs and benefits of a particular policy. Unfortunately, Tversky and Kahneman demonstrated that everyone, even professional statisticians, suffer from systematic biases in their intuitive judgements of probability. Students make a variety of identifiable mistakes when solving probability problems and even graduate students who plan to teach mathematics retain strong misconceptions.

The challenge is how to help students develop an intuitive grasp of these abstract concepts. We are particularly interested in combinatorics, a branch of probability that deals with the enumeration, combination, and permutation of sets of elements and their mathematical relationships, because it results in a combinatorial explosion: even simple problems result in hundreds of possibilities that cannot be represented simply with physical objects, virtual or otherwise.

Design Challenge

The original motivation for this project stemmed from observations of students in a university-level course in combinatorics. Faced with only paper and pencil, many had difficulty developing intuitions about probabilities and suffered from the ‘stereotype threat’ that they are poor in math. We hoped that letting students manipulate concrete objects while simultaneously observing the corresponding changes in deep structure, e.g. a probability tree, would reinforce their intuitions about the underlying mathematical principles. Our goal was to create an engaging and playful environment that avoids excessive mathematical notations and encourages discussion.

Combinatorix

Hardware

Combinatorix (Fig. 4) supports several input techniques: a camera detects the location of fiducial markers and a wiimote provides the position of multiple infra-red pens. A projector displays additional information around the tangible objects. The interactive surface is 60 x 45 cm. and can accommodate up to four students at the same time.

Figure 3. Combinatorix setup: The webcam detects location of fiducial markers; the wiimote detects position of infra-red pens

Software

The underlying application is written in Java and uses the Reactivision engine to detect fiducial makers [7]. Additional libraries, e.g., wrj4P50, communicate with the wiimote. The system is modular and can easily accommodate the creation of additional operators for constraining the sample space.

The current version displays two kinds of information: first, the tabletop interface shows a specific number of placeholders for objects. Letters can be placed on those spots to form a new combination. At the same time, the remaining number of letters for each step is displayed on top of each placeholder. A second screen displays a probability tree reflecting the current state of the problem. Letters can easily be replaced by other elements, including virtual, laser-cut and 3D-printed physical objects. Combinatorix supports up to 10 tangible objects and 20 virtual ones.

 Acknowledgments

 I would like to thank the Amir Lopatin Fellowship for funding this project; I would also thank Wendy MacKay and Paulo Blikstein for their support of this project.

Links

Project page on the TLTL (Transformative Learning Technologies Lab) webpage.

Publications

Schneider, B., Blikstein, P., & McKay, W. (2012). Combinatorix: a Tangible User Interface that Supports Collaborative Learning of Probabilities. ACM International Conference on Interactive Tabletops and Surfaces, ITS  ’12 (pp. 129-132). Boston, MA, USA: ACM.

Translating scientific findings into publicly understandable forms is an important step in making science accessible to the general public, thus is crucial in the public awareness of our environment, health and well-being. Visualization can be a powerful tool in achieving this translation. The goal of our research is to use interactive visualization coupled with predictive simulation as a fun and engaging informal learning tool for public informal science education. In this paper, we present WALDEN, an interactive visual simulation as a case study to examine whether a large multi-display multi-touch platform is appropriate for this type of visual informal science education. Our research focuses on the design of an interactive visual simulation system for illustrating and predicting the impact of climate change on the phylogenetic (evolutionary) patterns of species loss in Thoreau’s Woods in Concord, Massachusetts.

Design Requirements

Visualization designed for informal science education audience needs to be illustrative of the core scientific concepts, and also be self-explanatory for non-biologists. For our particular scientific story, we need to (1) present multiple related concepts and parameters including phylogeny, phenology, warming temperatures, and seasonality, and (2) provide visual illustrations such as imageries of flowering plants, geographic distribution of these plants, and the changing climate. To accommodate this large visual content requirement, we employed a multiple display environment that included a multi-touch tabletop (Microsoft Surface) and a large rear projected display wall of 12×6 ft² (3073 by 1536 pixels).

Two users interacting with WALDEN.

The WALDEN simulation seeks to model changes in abundance of a select group of plants found in Concord, as a response to climate change and human intervention in the local ecosystem. The stochastic simulation uses empirical plant abundance and climate change data observed in Concord over the last ≈100 years, to make future projections on the floral populations in this area.

The abundance change simulation is broken down into three modules: temperature shift (how individual species of plants are able to shift their flowering time (ft) to match fluctuations in annual average temperature; not evolutionary conserved), seasonality (how species are able to respond to fluctuations in temperature from year to year, measured as the standard deviation of average temperatures between years; phylogenetically conserved), and re-growth (the ability of plants to regenerate their populations to their initial equilibrium levels). Each of these modules acts as separate, individual processes that affect the change in abundance of the individual species of plants.

The two screen interfaces of WALDEN.

We carried out two rounds of iterative designs to arrive at the visualization and interaction described. The visualization in Figure 2 illustrates how simulated flower populations are affected by changes in annual average temperature and seasonality. People interacting with the simulation can alter the temperature and seasonality values directly on the multi-touch table, and the effect of these changes are presented on the detailed simulation overview on the large wall display.

Datawall: (A) Cross the top of the data wall, a dynamic scrolling graph displays the simulated annual average temperature (shown as a black graph line), which is superimposed over the simulated seasonality of that period (yellow band). The abundance values of the currently selected species are shown below. Both graphs use a logarithmic scale on the x-axis in order to visualize a longer time period. (B) On the bottom left is a radial phylogenetic tree of the 429 floral species from the Concord area [6]. The simulation only models a subset of these species, and these nodes are identified within the tree by a label presenting the name of the flower and its current abundance value. The currently selected plant species is highlighted in red. (C) On the bottom right, additional information about the currently selected plant species is presented: a map of the species distribution throughout the USA, a scientific drawing of the species, and an example photo image of the flower.

One interaction technique used on the tabletop.

We conducted a qualitative evaluation. The main objective was to assess potential learning gains. In total, 10 users (6 individuals and 2 pairs) participated in the study. We used a think-aloud protocol for prompting comments from individuals. At the beginning of each session, users read a short introduction on phylogeny, and the abstract of Willis’ paper. Our analysis suggests that every user grasped the main idea that closely related plants tend to react in a similar way to climate changes when using the system. Selected quotes from the participants illustrate this understanding: “I notice that similar species seem to be affected in the same way by similar stimulus”, “Species that are close through ancestors have similarities in the abundance behavior”, “Closer species were affected quite similarly by changes in temperature”.

We also noticed that the setup of tabletop/datawall effectively separated action and reflection: users tended to build hypotheses based on their existing pre-conceptions, make changes on the tabletop and then reflect on the results displayed on the large vertical display. Because users had to wait around 10 seconds to see the effect of climate change on the plants, they took advantage of this time to predict the plants’ behavior and elaborate alternative hypotheses.

References

Schneider, B., Tobiasz, M., Willis, C. & Shen, C. (accepted). WALDEN: Multi-Surface Multi-Touch Simulation of Climate Change and Species Loss in Thoreau’s Woods. ACM International Conference on Interactive Tabletops and Surfaces, ITS  ’12 (pp. 387-390). Boston, MA, USA: ACM.

Links

Project page (SDR lab, Harvard University)

Acknowledgments

Thanks to Matthew Tobiasz, who built the system; Chia Shen, who funded and supported this wok; Charlie Willis, who provided the scientific evidences on which this project is based; Laurence Mueller, for his moral support.

This project was part of a d.school class taught at Stanford (d.science). Here is the class blog. The goal of this project was to provide ways for students to share their level of understanding or confusion during a class lecture. We developed several prototypes of a phone-cliker which is a kind of “awareness tool”; students can see in real time how confused the class is. This facilitates action taking (stopping the professor, asking for clarifications) and community building. The following video shows how one of our prototype would work in a real classroom:

USER GROUP

Freshmen taking science classes (CS, math, physics…) in college.

EXISTING WORK

There is a general trend of ACTIVE / PARTICIPATORY learning for cognitive engagement in the classroom:

• Clicker / Personal Response systems widely used

• Also some clickers that communicate student understanding to teacher, but not necessarily to other students

• Nothing addressing the stigma of asking a strange question in class… nothing helping students to understand each other’s collective knowledge or to engage the teacher in cooperation

• existing systems are generally expensive; no low-cost solution to this need

Current work aimed at increasing teacher feedback is primarily focused on increasing student cognitive engagement. The intent of these systems is to keep students engaged by answering questions, with a secondary benefit of giving the teacher feedback on in-the-moment student understanding.
However, these systems fail to address two needs: the needs of students to feel safe and not judged for actually asking a question by raising their hand during class, and the need of the teacher for post-lecture data about student understanding.

EMPATHY OBSERVATIONS

Most of our observations come from a computer science class (“probability for computer scientists”). We observed TA’s office hours / lectures / interviews with students. Large lecture hall with hundreds of students. In situ, totally appropriate location and user groups. We were not intrusive. Students in lecture were watching and taking notes or surfing the web; sometimes students ask overly-advanced questions that aren’t helpful for the class. Students in office hours were working, getting help from the TA, or waiting.

SYNTHESIS

STUDENT 1 “There is a crucial moment when you learn a new formula, and you have to stop listening to the professor in order to mentally compute each steps.” “Lack of big picture; you’re just flooded with information and you need to recall as much stuff and procedures as possible.” “Video tapping is helpful because you can stop and do some consult other resources when needed.” “I love when the teacher says a joke after a complex equation, because it gives me time to conceptualize it.” “A huge cognitive load comes from the fact that you have to mentally replace the indices and variables in addition to knowing what they represent; writing their name in terms of the problem (concretely) helps a lot.”

TA “Students usually get it when I explain a concept to them; at least for a moment. The insight often doesn’t last.” “I try to make problems more familiar to students by using concrete examples, or by breaking it down in smaller problems.” “It’s difficult for students to generalize from one example; you change a small thing and they lose it.”

STUDENT 2 “The class is very useful but also painful. I feel like I’m really spending a lot of time on it.” “I need to look at the slides 3-4 times before class and again 3-4 times after class to feel like I have understood the lesson.” “Students have to learn to ask conceptual questions to the TAs; office hours have become sections because there is too much content in a lecture.” “It’s easier for people who are always doing maths; for others, there is a huge learning curve.”

OTHER STUDENTS “I never raise my hand.” “I’m slow, so it’s difficult to quickly come up with a good question.” “I’m okay asking questions 1 on 1, but I don’t do it even in small groups.” “Often I have a basic conceptual question and I don’t want to ask it because I should know the answer.” “It would be helpful to know what the rest of the class thinks.” Repeated by all students!

INSIGHTS

• Students generally aren’t comfortable giving direct performance feedback to the teacher, especially during class; they want to do it privately and anonymously to save face for themselves and the teacher.
• Students are more comfortable asking questions if they know that others have the same question.
• Students are more comfortable asking questions if they know the other students personally.
• It’s hard to speak up during a lecture for two reasons – classroom culture, and size / anonymity.

Points of View

POV 1

INSIGHT: People joke about not having time to write stuff down, but they don’t ask the teacher to slow down!

NEED: student needs a way to feel better about asking the teacher to slow down without pissing off other people or looking dumb.

POV: “I don’t want to be ‘that guy’ in class who is holding everyone back – I want to do something that helps me but is also good for everyone else.”

POV 2

INSIGHT: Teacher doesn’t realize that they’re going too fast.

NEED: Feedback from student. A way to create a classroom culture in which it’s okay to ask the teacher for things.

POV: “I’m doing the best I can for my students, and I don’t want any assumptions to hold us all back from creating a great learning experience.”

BRAINSTORMING SOLUTION

We want to come up with a way to have students loose their inhibition towards asking the professor to slow down, and this will probably happen if they realize they’re not alone in that desire. Let’s give students a way to discreetly let the rest of the class know that they feel rushed and want the lecture to slow down; clicker systems have been used for similar feedback before, so let’s make one with that goal.

PROTOTYPE 1

NEED: student needs a way to feel better about asking the teacher to slow down without pissing off other people or looking dumb.

IDEA: Give students clickers so they can show they want the professor to slow down; then show the dynamic results of this to all the students during the lecture, to show they’re not alone in that desire.

VARIABLE: Will students ask the professor to slow down if prompted by a passive indicator?

We asked students what they thought of this feedback mechanism and concept, but did not test it in a real lecture setting.

INSIGHTS

• Students liked and wanted to use the feedback mechanism; it was like a game.
• Students said they would be encouraged to give feedback, but if no one does, the professor still does not know about class confusion.
• If students have their phones out, will be tempted to goof off? Prevent students who leave the app from giving feedback?
• Is a generic indicator of “confusion” enough information to convey to students about their peers?
• What about students who don’t have smart phones?
• Students might down-vote difficult sections that are still necessary. They are not the best judges of learning content.
• Feedback isn’t constructive. This puts a lot of demand on the teacher to be a good performer; will this frustrate teachers who get bad feedback?

PROTOTYPE 2

NEED: teachers need to refer to previous feedback to identify complex concepts or slides that are confusing, since students are not giving the lecturer the feedback in-class.

IDEA: Build an histogram of students’ confusion (updated over time as class goes on).

VARIABLE: Will the lecturer use the immediate feedback to help with explanation?

Based on insights from previous user testing, we choose to improve our previous prototype “clicker” system to include a way to explicitly give the professor immediate feedback. Students still are able to press a “help” button on their clicker or phone and a program records when during the class this button was pressed.

We show the professor, in lecture, a graphic that shows “student confusion” as a function of time. They can see how well understood their comments just were, and it allows them to go back after class and look at what parts of their presentation need to be clarified and improved, by looking at what timestamps had the most confusion.

We tested this system in a small lecture given by one of David’s labmates. The lecturer could see a plot of when during his talk students were having the most confusion; the audience had no feedback, other than knowing when they themselves pressed “help” on the clicker page.

Histogram of students’ confusion during lecture:

Line chart directly next to the slides:

INSIGHTS

• Audience felt guilty about using clickers: “I should just raise my hand, not use this thing.” Even after one testing session, we might be changing classroom culture!
• Attendees did not use clickers very much. A class-facing component seems to be necessary to keep faith and engagement in the system.
• Presenter did see chart clearly, and tried to act upon the new information.
• “I didn’t know what to re-explain.” Presenter though that this would be more useful after lecture, when he would have time to re-assess the specific trouble spots.

• Prototypes need to be tested in environment that mimics the intimidating environment of a large intro class; we were in a smaller, optional lecture, that was very low stress.
• Presenter might need training as to how to deal with immediate feedback usefully.
• The system needs both prototypes simultaneously to be functional; lecture environment can be changed more if there is an audience-facing screen, lecture confusion can be reduced more if there’s a professor-facing screen.

PROTOTYPE 3

NEED: Teachers needs to be able to identify what was confusing during their lecture (a histogram may not be enough); name of confusing concepts.

IDEA: Allow students to directly type a keyword on their phone / laptop

VARIABLE: Will students be willing to document confusing concepts? Can the lecturer incorporate current confusing topics into their talk in real-time?

We had previously ignored how the lecturer would react to this feedback, but now that our last prototype gave us specific feedback on that, we decided to incorporate more informative feedback into the system. We still wanted to keep the mechanism by which the students asked for help very straightforward, so we show a list of most frequent topics on people’s phones and allow students to easily type in new topics at the bottom. The current most requested topics are shown on the presenter / class-facing screen along with the previous histogram data.

We were unable to find an appropriate lecture that was willing to let us test out our clicker system before the end of the quarter; based on our previous testing, we knew that we needed to find a class that was as close as possible to the original lecture situation we performed needfinding on. Unfortunately, the larger lecture classes were unwilling to introduce things that might delay class this late in the term. We instead asked people and presenters for feedback on this updated system.

INSIGHTS

• Presenter still was able to interpret data, and said he could incorporate it into his responses easier.

• Presenter said that this would be especially helpful in reviewing how a presentation went after the fact, and knowing what topics to focus on / review in later lectures.

• Similar to how people use hashtags during conference talks on Twitter; this shows the principle could be well received.

• Now that it’s possible to directly and anonymously communicate with the professor, are we preventing classroom culture from changing? Will people be less likely to ask questions now that students are aware the professor already knows their question?

FINAL THOUGHTS

Testing that any device truly changes classroom culture will be difficult and require a longer term study than we were able to complete. We are still hopeful that further testing and answering some of our outstanding questions on the interplay between social norms between student and teacher, and between students will let that goal be achieved. The fact that after one testing session one student already said, “I should just raise my hand, not use this thing,” leads us to think it’s possible. In a sense, we’d like our project to be self-destructing; we think that any system that encourages students to ask questions and succeeds will positively reinforce the act and, hopefully, will make itself unnecessary.
Team: Daniel Green, David Selassie, Bertrand Schneider

Motivation: Wouldn’t that be great to instantaneously visualize past events that happened exactly where you are? With current technologies, this is now possible. The GPS of your phone can be used to link your position with a database loaded with important historical events related to your location. We believe that this kind of casual, on the go learning is being facilitated with the technological advances we are witnessing.

More specifically, the tracking of movement across place and the embedding of narrative within place enabled by GPS-enabled phones and object-aware systems offers the opportunity to collect data related to fundamental questions raised by philosophers for thousands of years about how humans create, perceive and interact with their worlds. Social scientists have been interested in how cultures construct a sense of place, and the role that place plays in meaning-making, cultural cohesion, and sense-of-identity within communities. Examples include Keith Basso’s influential work on the role of place-based storytelling in inter-generational knowledge sharing in Apache culture; the role of place-based narrative in the maintenance of national identities engaged in conflict (e.g., Israeli and Palestinian national identities formed in relation to the city of Jerusalem), and identities of displaced or re-placed peoples such as refugees and migrant workers. These issues are not limited to those who are marginalized; as our work lives become increasingly mobile, flexible and virtualized, there is common awareness that the human sense of place is changing. However, untill recently researchers lacked access to large datasets related to people’s movements, real-time sense of place, and production or consumption of media about place-based experiences – instead having to rely on post-hoc self-report measures such as interviews and surveys. Research has shown that the cognitive and social resources embedded in places significantly influences what we think about, how we learn, and what we choose to do and learn next. While research shows that place is a large factor in the formation of social identity, little is known about how transition among places influences learning, and how geo-locationally aware tools might change or enhance perceptions of and movements through places. For the first time in history, we have the possibility of accessing the movements and real time place-based thought and learning patterns of communities.

TimeExplorer provides the kind of functionality described above. Students are able to retrieve stories related to their location on their IPhone; they can also post new stories and comment on them. See the screenshots bellow for more details,

As well as a demo of the app:

Research Team: Bertrand Schneider, Shima Salehi, Sarah Lewis, Roy Pea

Status: the app has been removed from the app store due to the expiration of my developer licence (16 Feb 2012)

Publications: none.

This project was part of the Beyond Bits and Atoms class taught at Stanford by Prof. Paulo Blikstein. We had to find a child, interview him/her, discover what their dream toy would be and build it. Bellow I describe my process and various iterations of prototypes:

My interview with the kid went well, I tried to alternate between asking her questions and playing with her. We spend one hour in her room, where she described me all her toys and how she played with them. She loves zebras, and had a big collection of them: big, small, soft, hard, etc. Her dollhouse has even been converted into a zebra house, and the zebras actually sleep in standard beds with humans.

She described me the toy of her dreams as a box where zebras and giraffes could race against each other. Her design included several elements:

• a field of grass where the animals could race
• she wanted to have 10 zebras and 10 giraffes
• the victorious animal would win a zebra cup
• she mentioned teams that she could switch so that the other animals could rest

We conducted this part of the interview in the living room, where her mother helped her describe the toy she wanted. It was actually very helpful to have one of her parents with me, because information flew more easily between them (and obviously the mother knew more easily what the kid meant). She also drew several sketches for me: 

she decided to draw this zebra and giraffe so that I could use them for her toy. She also drew a field of grass, with the starting point on the left and the finish line on the right:

Here is the part where she describes me her toy:

I tried to follow her instructions as closely as I could but I quickly realized that I wouldn’t be able to provide her 10 zebras and 10 giraffes. My first (and biggest) challenge was to make the animals move in the running lanes. My first attempt involved a treadmill coupled to a gogo board:

That didn’t work so well, because the paper would easily slide outside the wheel. I could have solved this problem by cutting and gluing some acrylic circles to the sides of the spools; however the whole system was too fragile and too complicated to build. So I changed my mind and tried to use rubber bands to propel the animals:

This first prototype didn’t work very well, mainly because the wheels were too small and the force wasn’t evenly distributed over time (the system above would use all the force in one shot). With the help of a TA, we designed another iteration that successfully propelled our car frame:

The rubber band is attached to the frame of the car on one side, and on the axis of the wheels on the other side. By moving the car backward, the rubber band gets twisted around the axis and accumulates force; by releasing the car, it discharges its tension.

The final design of the boxes involved a zebra texture on the outside and multiple decorations on the inside: some seats so that other animals could watch the race, a zebra cup after the finish line and an inscription on the ground (“zebra race”):

Here is the final video of the project:

The kid was very happy with it, and we spent quite some time playing with it afterwards. Among other remarks, she said that she didn’t think that I could make the toy that good, and that the box was “the goodest toy she ever had”.

Here is Salma discovering her toy:

Bertrand

In this project we discuss the design and evaluation of a gaze-based interaction technique for visualizing a phylogenetic tree. In our implementation, users were able to brush, expand, collapse and rename nodes of the tree by looking at the species of interest and using the keyboard. Our goal was to investigate the effectiveness of gaze-based interactions for a specific task (e.g. exploration of tree structure).

we tried different visualizations and discovered that natural and efficient gaze-based interactions are difficult to implement. Our original idea was simple – to use gaze to facilitate users’ perception of a graph. We originally implemented three prototypes enhanced with gaze-based techniques: a scatter plot, a bar graph, and a choropleth map. We wanted the detailed information to be revealed when users focused on a certain region, whereas things in the periphery would stay hidden. We soon realized that this approach was not ideal for users, as having the detailed information shown on a single region prevents them from comparing or analyzing content on other parts of the visualization. Moreover we observed that having the visualization radically change based on the gaze’s location is highly distracting. We concluded that users would prefer to have everything displayed because they are able to choose what they want to see without bothering moving their gaze.

We compared this approach with a purely mouse-based interface. We found the following results:

Users were significantly slower to perform the “brushing”, “navigating” and “analyzing” task when using their gaze. Users were equally fast for the “renaming” task when using a mouse or their gaze. There are several reasons for this pattern of results: the inaccuracy and latency of the eye-tracking technology we used, as well as a certain discomfort for users to control things with their eyes.

Even though eye-tracking techniques can potentially compete with traditional input devices, users still consider those interactions as more demanding. It is not clear whether this feeling would decrease over time with appropriate training. In conclusion, we believe that eye- trackers should be used in conjunction with other interactions techniques for simple tasks (such as switching between windows or selecting a text field). Trying to perform complicated tasks with the gaze is similar to control the rhythm of our breadth: this is a conscious and effortful task. As a conclusion, designing efficient gaze- based interaction is complex and difficult; researchers have not found yet a situation where eye-tracking techniques offer a real advantage over traditional input devices.

As future work, we are interested in developing less disruptive interaction techniques for eye-trackers. More specifically, actions that don’t require visual change on the screen.

For this study I explored the perceptual benefits of using tangible objects for a spatial problem-solving task. In two experiments, thirty-three participants completed the Paper Folding Test (Ekstrom, French & Harman, 1976) with either a physical or abstract representation of the material:

We found that subjects (females, in particular) were faster and more accurate in solving the test using physical material.

One interpretation is that physical material allows for “epistemic actions”: namely, “actions whose purpose is not to alter the world so as to advance physically toward some goal (e.g., laying a brick for a walk), but rather to alter the world so as to help make available information required as part of a problem solving routine. Examples of epistemic actions include looking at a chessboard from different angles, organizing the spatial layout of a hand of cards). In our case, women in particular took advantage of this feature of physical models. Another interpretation is that females may be more likely to rely on surfaces features, which is a double-edged sword for a spatial task. Indeed, when the representation is close to reality female participants don’t have any trouble to solve the problem at hand; however, when the surface features are confusing they have much more difficulty figuring out the right solution. As a consequence, we can imagine that training some subjects to extract the deep structure of a problem to help them perform better at this task.

Implications for Education

This study has several implications for education: for example, teaching chemistry usually involves working with complex 3D molecules. It would be interesting to explore whether graphic displays of molecules that include more perceptual clues such as shading would enhance the level of performance in chemistry. We also propose that some students (females, in particular) may benefit form a customized curriculum and potentially be trained to transition from a physical to an abstract representation in order to extract the deep structures of a problem. Finally, future studies should also test whether it is possible to teach students to extract the deep structure of a problem and rely on this mental model to perform a task. Indeed, it may be possible that those skills 1) are innate, and thus cannot be changed; or 2) necessitate en extensive training (e.g. several hundreds of hours) to be improved.

Future Work

I am currently replicating this study using an eye-tracker. The goal is to more finely detect participants’ strategies and describe patterns of failure and success.

The goal of this project was to facilitate students’ transition form concrete experience to abstract knowledge. We chose to work with students in logistics because they usually spend a lot of time on the field (working in warehouses) and little time in school. It is difficult for them to connect their everyday experiences with concepts taught in school. The Tinker Table is a possible solution to this problem; this is learning environment that simulates a small scale warehouse where students can more easily transfer and communicate their practical knowledge to a classroom setting.

This learning environment has been created at the CRAFT in Switzerland. Pierre Dillenbourg and Patrick Jermann have been supervising this project; the software behind the Tinker Table has been created Mainly by Guillaume Zufferey. My contribution was to design and run an empirical study comparing a multi-touch and tangible version of the Tinker Table. Here is a movie showing the Tinker Lamp being used in a swiss classroom:

Before running the study, I explored how different constraints influenced the way students would solve a typical problem in logistics (e.g. maximize the number of shelves in a limited space). Among other things, I varied the size of the representation, the “physicality” of the shelves, I had students solve the problem with traditional means (e.g. pen), on different surfaces and sizes:

I found that most of the time participants benefited from using physical objects. Working on large areas was also problematic, because moving things around and erasing shelves took more time than on small spaces. All those observations inspired the study that I ran comparing multi-touch and tangible interaction with the Tinker Table.

I then conducted a study to investigate the role that tangibility plays in a problem-solving task by observing logistic apprentices using either a multitouch or a tangible interface. Results showed that tangibility helped them perform the task better and achieve a higher learning gain. In addition, groups using the tangible interface collaborated better, explored more alternative designs, and perceived problem solving as more playful.

Mediation analysis revealed that exploration was the only process variable explaining the performance for the problem-solving task.

For more details on the project, please consult the webpage of the CRAFT.

Acknowledgements

Prof. Pierre Dillenbourg, for giving me the opportunity to work in his lab and contribute to this project; Patrick Jermann, for teaching me so much about the field of Computer-Supported Collaborative Learning and for his previous advice on how to design this study; Guillaume Zufferey, for building this amazing learning environment and supporting me during my time at the CRAFT; all the doctoral students (Son, Quentin, Andrea, Simon, Mirweis, Olivier, Kahleb and all the others) for their general intellectual contribution and great Friday “apperos”.

Links

Project page on the CRAFT website (Ecole Polytechnique Federale de Lausanne).

Publications

Schneider B., Jermann P., Zufferey ., & Dillenbourg P. (2011). Benefits of a Tangible Interface for Collaborative Learning and Interaction. IEEE Transactions on Learning technologies.

Jermann P., Zufferey G., Schneider B., Lucci A., Lépine S., Dillenbourg P. (2009). Physical space and division of labor around a tabletop tangible simulation. CSCL 2009.

The goal for this project was to create a collaborative learning module that allows students to experience the process of creating and interpreting phylogenetic trees. In discussions with instructors of college-level evolutionary biology, we discovered that the concepts behind tree-building can be difficult to grasp for students, and the ability to correctly interpret the information communicated through trees often does not come naturally. By implementing a collaborative activity that utilizes new ways of interacting with trees and tree-building, we tried to address some of these challenges.

We implemented this computational learning module on the Microsoft Surface platform. This allowed us to explore the use of interaction techniques that would not be possible with a traditional pen-and-paper exercise. Students can physically order and arrange character traits and explore different arrangements and representations for the taxa, transitioning from one visualization to another with ease. The following video shows the different steps of our learning scenario:

The Phylo-Genie environment presents users with a scenario that motivates the learning activity. Participants ’travel’ to Australia as researchers to assist in data collection and analysis. The trip is executed as a series of nine stages. During the trip, a user is bitten by a venomous snake. The users then have to choose between 4 equidistant hospitals; each hospital has only one type of anti-venom. They have time to reach only one hospital before the venom irreversibly affects the bitten user. Since the species of the snake is unknown, participants must learn tree-building techniques to assess the common ancestry of the Australian snakes. Closely related snakes share the same venom (and thus anti-venom). Successful tree construction and interpretation allows users to select a treatment. This scenario, anchored in a real life setting, provides participants with motivation for learning phylogenetics.

To evaluate Phylo-Genie’s strengths and limitations in supporting collaborative learning of phylogenetics we conducted a between-subjects experiment with 56 undergraduate and graduate students. We compared the system implemented on a multi-touch tabletop to a traditional pen and paper implementation of the Phylo- Genie scenario. The rationale for this choice was to conduct an ecologically valid comparison of our system as pen and paper is the premier media used for teaching in college settings. In this study, we examined the similarities and differences of the two implementations in terms of quantitative performance and qualitative behavior.

More specifically, four guidelines shaped the design of our system: firstly, we focused on developing an activity that engaged and motivated students. Interactive technology helped reach this goal. We created rich and viscerally compelling contents that were not available in paper format (e.g. immersive videos, interactive contents, animated tutorials). The engagement questionnaire suggests that students felt more involved, viewed the learning experience more successful, and rated the task as being more aesthetic. Secondly, we supported collaboration through physical tokens and tabletop implementation embodying the main actors of this scenario. The goal was to reinforce users’ ownership of both physical objects and areas on the interactive surface. We interpret the more balanced division of labor between users as an indirect evidence for this hypothesis: in the tabletop implementation, participants took more turns and spent more time collaborating (e.g. pooling information, coordinating activities) rather than working independently. The results are supported by other studies showing that territoriality plays an important role in tangible learning environments [27,44]. Thirdly, we dedicated a significant part of the learning activity for students to reflect on the content. Even though the paper activity also offered this opportunity, we argue that switching from a very active role (interacting with physical tokens on a multi-touch surface) to a more reflective stage (articulating discoveries as formal concepts) provided better opportunities for understanding and refining phylogenetic concepts. Fourthly, we tried to provide a sense of control and autonomy to our participants by building automatic feedback and scaffolding in our activity. Although we did not device any measures for this dimension, we believe that this principle played a major role in the learning process; in traditional classrooms, students often have a passive role and depend on the teachers for advancing in the domain taught. In Phylo-Genie, users control the pace of the activity and decide when to move to the next step without any social pressure. Future studies should develop metrics to measure this construct and investigate how it helped students’ learning.

In summary, the four guidelines helped develop a strong learning activity. Additionally, we observed that the learning process took an indirect path. The mediation analyses revealed that collaboration was the strongest factor in supporting knowledge-building and that engagement acted as mediator for a productive collaboration. Thus, Phylo-Genie succeeded in supporting learning by increasing users’ collaboration, which in turn was improved by a high engagement. It suggests that interactive environments can and should support learning in very diverse ways.

Project members: Bertrand Schneider, Megan Strait, Sarah J. Elfenbein, Laurence Muller, Orit Shaer, Chia Shen

Publications: Schneider B., Strait M., Muller L., Elfenbein S., Shen C., & Shaer O. (2012). Phylo-Genie: Engaging Students in Collaborative ‘Tree-Thinking’ through Tabletop Techniques. CHI 2012.