Designing a questionnaire to quantifiably measure user values (2)

Finally the panel of 11 experts have all sent their responses back regarding the content validity of our proposed questionnaire in here. We ended up with 7 values with at least one relevant item. For an item to qualify as relevant to a value, 9 out of the 11 as per this method need to have ticked the corresponding box in the matrix (as either essential or useful). You can see in this table the final result, i.e. the matrix of questionnaire items (in Dutch) and which values are supposed to be relevant.

Designing a questionnaire to quantifiably measure user values

Well I’m developing a questionnaire now! It would be quite the first of its kind to “quantify” values, or how far a certain value is fulfilled anyway, for a certain user. We have chosen a number of values that we felt are relevant to the research domain, including, using Rokeach’s terminology (1973):

  • Social recognition: to be (mainly, as a child) socially involved and accepted by other children.
  • Family security: that the members of a family are safe and free from harm.
  • Independence: to be (mainly, as a child) capable of doing certain things without constant supervision from elders.
  • Freedom: to be (mainly, as a child) capable of doing certain things without explicit permission from (or consultaion with) elders.
  • Inner harmony: peace of mind and having no worries especially about children, and other family related issues (mainly as a parent).
  • A comfortable life: to have (mainly as a parent) less trouble in life, especially with raising children.
  • An exciting life: (for both parents and children) to have a life filled with interesting (mainly social) events.
  • Friendship: (mainly taken from a child’s perspective), to have valuable friendships with peers.

Currently creating items that are supposed to be able to measure some of these values. They will be sent out to a panel of experts for a Content Validity Analysis, so we can decide which items go and which items stay, in terms of each of these values. More on this to follow!

Focus groups in our target area

We have lately performed three focus group sessions with our target group members in South Holland, which was composed of two groups, the first group of six parents, and the second group of six of their children.

Through a small “snowball sample” we requested these groups to participate in the studies. Our snowball sample started with a contact who participates in the school board, a youth centre and in a website for the local community.

The first focus group session included the six parents only. We introduced them to our project, research, and explained the aim of our user studies. To stimulate discussion, we displayed a few usage scenarios and design claims (i.e. claims about a few positive and negative effects within our scenarios) then asked the participants to rate to what extent they agree with our claims. See slides at the end of this post. This was the session when we provided the parents with the cultural probes kits.

The second session (three weeks later) included the same group as the first session. The parents brought back the material they (along with their children) collected during that period using the cultural probes kit, and then proceeded (individually) to describe the data (e.g., pictures, map highlights, etc.) they collected with their kits. This process stimulated the discussion for a further 45 minutes in which many of the parents and their children’s life issues, values, and concerns were raised.

The third session included the six children only. The ages of the children ranged between six and eight years old. That session was led by an experienced elementary school teacher, and consisted of a discussion where the teacher asked the children a number of open ended questions related to their knowledge and usage of current technology, what activities they are allowed to do, how they connect with other children at school, sport clubs, and other places.

All sessions were audio-taped.