Website for conflict resolution user study is now online!

We have at last finalized the website for the conflict resolution user study mentioned here, and it is online now for anyone to participate. We have also launched a campaign on Microworkers.com to add more participants. You can check it out at:

http://ii.tudelft.nl/~alexk/ConfRes

Don’t forget to watch the instructional video(s)!

Advertisements

Experiment: can human values be used as predictors for solving user-created normative conflicts?

Motivation and objective

Previously we showed that social commitment models could play an important role into making social applications more adaptive/intelligent, and better promote user values in that domain. However, social commitments are norm-based, conflicts may occur and they need to be resolved.

Many conflict resolution approaches exist in literature, but they are rather concerned with the “how” rather than the “why”, i.e. how to implement a certain conflict resolution policy, rather than what that policy contains. We are proposing that, given a user’s value profile (i.e. order and/or preference of a number of relevant values), and how that user believes certain commitment are relevant to the same set of values, that we can predict the user’s preferred solution if a conflict is to occur involving any two of these commitments. The importance of this is that, assuming we were able to predict the correct solution for a majority of conflicts, that social media platforms can use contextual data to automatically solve conflicts between privacy/newsfeed settings created by the user in the future.

Relevant values

Using Rokeach’s value survey as a basis, we select a sample of potentially relevant values. This is not meant to be comprehensive list of all values possibly involved. The interpretation of these values here is generalized, and participants may as well project their own interpretation over the ones provided.

• Safety (Rokeach: family security)
For one or members of one’s own family to be safe from dangers or harm.

• Independence/freedom
For one, or members of one’s own family to be capable of doing what they need to do on their own, without being dependent on others.

• Social recognition/friendship
For one or members of one’s own family to build true friendships, a social life, and be recognized or distinguished amongst others in their social circle.

• Privacy (not in Rokeach’s list but relevant)
For one or members of one’s own family to have important information about them only shared with those they agree to share it with.

• Responsibility
For one or members of one’s own family to act responsibly in situations where that is required.

• Peace of mind (Rokeach: inner harmony)
For one or members of one’s own family to live with few or no worries and/or disturbances.

Hypotheses/research questions

H1: users’ value profiles in addition to how they believe certain commitments relate to their values can predict/explain the preferred conflict solution.

RQ2: will users have preferences regarding conflict solutions that deviate significantly from the “no difference” point?

RQ3: Is there an association between the agreement type (i.e. its elements) and the value profile associated to it? (E.g. location and privacy)

RQ4: users will have no significant preference for one value over another in their value profile.

…to be continued.

 

Simulated Work Tasks: an online study of usability and contribution of a Social Commitments model

About to run a Simulated Work Tasks (SWTs) experiment using the online platform microworkers.com. The goal of this experiment is to assess the usability and domain contribution to the social commitments (SC) model that we created. Participants will perform 4 tasks each, and at the end of each task, they will try to solve the family-life domain problem through creating an agreement using a menu representation of the SC model. After that, they will have to rate how well did the options in the SC menu contribute towards the solving the problem in the scenario, using a slider.

You can try it out yourself as well, using this link. Make sure you see the explanation video first!

App user test @De Lange Keizer, Delft

Description of the first of a series of user studies for our app, this one conducted at De Lange Keizer daycare center, Delft. Eight children participated in the two testing sessions.

This user study was the first field test of our implemented app prototype. The app is a location sharing device for families with children in primary school (between 6-12 years of age), which allows users to create locations, check-in in these locations, share and receive check-ins from others as they choose, and create normative-based social commitments with other users regarding sharing/receiving check-ins. We believe that this additional capability (social commitments) will provide a better support for user values for children, namely social recognition, friendship, independence, and freedom (amongst other hypotheses).

We have built two editions of the app. Edition 1 comes without social commitments (a.k.a. afspraken) and edition 2 comes with social commitments. These hypotheses/research questions we are attempting to answer in this user study are:

  1. Within the domain of family life, e2 will provide better overall value support for children than e1.
  2. Within the domain of family life, e2 will be perceived more as social actor than e1.
  3. Within the domain of family life, e2 will be perceived more as useful/usable than e1.
  4. (Research question) what subset of the possible social commitments will our users use, and for what purposes/values?
  5. Children will expect that the presence of an app in their life will increase support to their values.
    1. Sub hypothesis: the children will expect that the presence of an app in their life will increase their independence (or personal freedom).
    2. Sub hypothesis: the children will expect that the presence of an app in their life will increase have a positive effect on their friendships (or their social recognition).
  6. Children will expect that the presence of an app in their life will increase support to their activities.
  7. Children will expect that the presence of an app in their life will remove some of the limitations they have.

Setup

The user study took place at a local day care center (Dutch: buitenschoolse opvang) in Delft. This day care center is a place where children are brought after school is over, until their parents pick them up at the end of their workday. It contains 9 different activity rooms where children can freely participate in activities, such as a painting room, music room, playground, lunchroom, etc. The day care center officials have assisted us finding children who wanted to participate in the user study.

Due to the number of participants (8 children, aged between 7 and 10 years) we opted for a within subject, counterbalancing design, meaning that children were randomly split in two groups of four, each group testing a different edition of the app in each of the two sessions.

Introductory session

We held an introductory session one week before the user study sessions, where the researchers met the participants, introduced the app using a tutorial video, and answered their questions about the app and the procedure of the user study.

User study sessions

The two sessions were identical, and they were one week apart. Below is a description of what happens in each session.

Children had to perform “missions” using the app. A mission is either:

  • Instructional, directly pointing to a functionality in the app to help the user get acquainted with the app more. Examples: checking-in in a certain location, adding other users to their friend or family list, and creating a social commitment.
  • A story-lined, simulation of a real life situation, that requires the user to figure out what needs to be done with the app without direct instructions. Examples: go to the lunchroom. Did you find any of your friends there? How can you ensure that one of your friends will inform you when they enter this location? Check who’s in the painting room. Anyone from your team? If yes, try to tell all your friends that you are both at that location. Can you see where others on your team are? Try to go out and find one of them. Hint: check your event list.

There were 37 mission cards in total (17 instructional missions, 20 real-life situation simulations, see example in photo below).  The missions had an even distribution of “envisioned” solutions in terms of app functionalities and the possible types of commitments that need to be created.

Children would pick random missions from the pile, read the text, (attempt to) perform them, and then bring the mission back when they would like (whether finished successful or not) to the pile to pick another one, and so on. The user study stopped when one hour had passed in each session.

During which the first 10 minutes only instructional missions were available to be picked from the pile (in order to build some app knowledge). Afterwards, we added the rest of the missions to the pile.

Collected data

  1. Value measuring questionnaire: answered by children at the end of sessions 2 and 3. Contains questions to measure fulfillment of the values Friendship, Social recognition, Independence, Freedom. (Hypotheses 1,5,6,7). Questions in this questionnaire were framed into a hypothetical future tense: e.g. “if I had this app in my life, it would <less easy………..nothing will change……..much easier> to find where my friends are. Before they answered the questionnaire, children watched a short video explaining how the questionnaire should be answered, with a period for questions and answers.
  2. Questionnaire for usability and social actorship, also answered by children at the end of sessions 2 and 3. Contained questions for usability, liking, dominance, trust, and intimacy. (Hypotheses 2,3).
  3. Every time a child picked a mission from the pile, the id numbers of the child, mission, and timestamp were recorded. (Research question 4)
  4. Behavioral sampling: every 10 minutes, one of the researchers recorded every child’s emotion/body language according to a predetermined coding schema (derived from Markopolous, Evaluating Children’s interactive Products, pp. 182, with positive codes created by reversing the negative ones already existing in the text), their location, and their engagement with others, i.e. whether in groups or alone. (Hypotheses 1,5,6,7).
  5. All app interactions with any user were recorded and time-stamped on the server, e.g. every check-in, every time a user added another user to a list, every time a commitment was created, accepted, rejected, etc. (Research question 4).

missions