Mapping the lexicon through a word association game.

Background

Large online projects that measure what words mean in the community, such as the ongoing Small World of Words (https://smallworldofwords.org, SWOW) use a word association task to investigate how semantic cognition is shaped by language and our experiences. However, existing datasets like SWOW are often unbalanced: most English speakers who participate online are American, most volunteers are women, and little information is collected about factors like political orientation. This makes it difficult to study how meaning differs across groups. Our project addresses this by collecting new data from men and women in Australia and the USA. A second goal of this study is to determine how different instructions and types of feedback influence the responses people give. By comparing “free” associations with versions of the task where participants receive feedback allows us to test how feedback changes how people approach the task, and what the trade-offs are when using games-with-a-purpose.

Research Questions / Hypotheses

How does word meaning vary as a function of gender, geographical location and political affiliation? How common are these differences and to what degree are they related to lexico-semantic properties of the words, such as their concreteness, semantic category and emotional valence? How do different types of feedback affect responding and study engagement compared to situations where no feedback is given?

Participants

A total of 901 participants participated in the study, with the majority of our participants were native English speakers. Exclusion criteria were applied based on who finished the task or no, whether most words were known English words and what the participants' native languages were.

Methods

For each participants we collected demographic data (age, gender, education, location, nationality, languages, vocabulary knowledge, political affiliation, years residing in the US vs Australia) and word associations and reaction times for a list of cue words. We will also included a short survey about the participant's experience of the study. The stimuli will consist of common English words and include items that men and women might represent differently (e.g. “girlfriend”, or, “party”). It will also include words that might be different across the political spectrum, such as “freedom” and “migration”, and words that potentially differ between the US and Australia are “cricket” or “lollies”.

Results

Data collection for the American part of the study is currently ongoing. Some preliminary analyses have been performed on the Australian data (REP). For each cue word, we tallied the number of distinct responses and clustered them based on semantic themes for both men and women. Unlike previous work that relied on smaller samples, the association frequencies were highly reliable. Our findings show that there is significant variation in meaning across gender (over 20% of cases) and that gender-differences extended beyond those words where we expected to observe them (e.g. "husband").

Implications

Unlike previous work that relied on smaller samples, the association frequencies were highly reliable, which opens up a range of possibilities to study individual variation if sample size requirements are met. Our findings show that there is significant variation in meaning across gender (over 20% of cases) and that gender-differences extended beyond those words where we expected to observe them (e.g. "husband"). Feedback was found useful and had a positive effect on participant engagement, leading to higher levels of enjoyment and willingness to volunteer in future studies. However, the nature of responses was somewhat different between no-feedback and feedback conditions, and further work needs to establish how to manage these trade-offs. We plan to integrate these findings on platforms such as Small World of Words (https://smallworldofwords.org/project) and make the data and results available to both researchers and the community. The final results will be presented at conferences (e.g. Cognitive Science) and submitted as a journal article. In a second stage, we aim to extend a shorter version of the study to the community. In the mean time, you can also contribute to similar studies such as SWOW (https://smallworldofwords.org/).