Exploring personal data and AI as resources for self-examination

Timeframe

2018 ↝ 2023

Keywords

  • Speculative Design
  • Metadata
  • Research Through Design
  • Interaction Design
  • Digital Archives
  • Artificial Intelligence

Outcome

  • Co-speculation sessions with 17 introspective practitioners

  • 7 speculative Introspective AI products

  • Publications and presentations at ACM DIS 2021 & ACM CHI 2023

  • Masters thesis


The Core Idea

Introspection is the practice of looking inward and examining one’s own thoughts, emotions, values, and desires. It is different from simply reflecting on memories; it is the critical practice of assessing past experiences and patterns in one’s life and asking questions about what has been achieved and what one wants for the future .

This project investigates how Artificial Intelligence (AI), combined with the archives of personal digital data that increasingly mediate our everyday lives, might act as a resource for introspection. We explore how AI can generate alternative perspectives on one’s life and surface paradoxes that this might raise.

Background & Framing

As interactive technologies and personal data archives continue to grow, people’s practices of self-reflection are expanding. Prior research in HCI has shown how photos, music, social media, and location traces can support reflective or nostalgic experiences. Yet, considerably less research has focused on the specific practice of introspection—the active confrontation of one’s values, biases, and future aspirations through engagement with personal data.

AI presents intriguing possibilities in this space. While existing commercial applications often use AI to guide mindfulness or organize journaling, these services rarely embrace the complexity and ambiguity of people’s digital records. Our work instead asks: how might AI operate as a design material to create interactive, evolving resources for introspection?

Introspective AI Design Proposals

To investigate this question we first developed a range of different speculative product concepts that are framed within a fictional AI company called Meta.Aware. This products are: Everyday Personality, Music Reflection, Mind Probes, Vision Shrine, Hello, Cyber-Self, Dream Streams, and Deep Talk Report. We then progressively refined these concepts into seven different short videos to explain them.

Meta.Aware is the fictional company that all of our speculative Introspective AI products exist within. Watch the video below to find out more about Meta.Aware:

Everyday Personality presents a chatbot interface that delivers short introspective prompts. This Introspective AI service uses its deep understanding of your behavioral data to intervene in everyday life with tailored short introspective prompts delivered at opportune moments:

How could such personal data records offer a resource for supporting situated experiences of introspection over time? Music Reflection explores this question through an application that integrates within Spotify and generates short introspective prompts:

Mind Probes is a smartphone app that works in tandem with external hardware sensors: sound, color, smell, haptic and vision. It prompts the user to collect sensory stimuli from the material world that reflect social and emotional associations—connecting inward associations with encountered phenomena. Mind Probes encourages introspection through long-term activities akin to a scavenger hunt that that supports collaboration and alignment through open-ended experiences:

The Vision Shrine device visually manifests a user’s goals, dreams, and desires as data collages—an ideal self-canvas that updates in real-time as it consumes their personal data. Drawing on confrontation and alignment, the Vision Shrine changes the scale of goals depending on how they are being prioritized in everyday life, as an ongoing dialogue between a user’s lived reality and their ideal self:

Hello, Cyberself offers a conversational window into the assumptions (and biases) that a personal Introspective AI has developed over time. It leverages real-time voice cloning technology to speak to you in your own voice. It expresses introspective prompts to you as you— embodying your personality traits and beliefs, and then reveals the data ‘under the hood’ that generated these inferences:

Dream Streams speculates on how sleep and active recall of dream experiences could be mobilized by an Introspective AI to generate new open-ended and guided introspective resources. Dream Streams combines a dreamcatcher-like device paired with mobile applications to offer windows into one’s subconscious and open new pathways to self-awareness:

Deep Talk Report is an application that audits verbal and written conversations to find and classify deep exchanges. These analyzed accounts are curated guided introspective sessions and are also woven together to generate broader thematic reports, which confronts users with emerging patterns over time. Themes are further explored through the contextualized introspective activities that are proposed in each report:

Co-Speculation with 17 Introspective Practitioners

We explored these concepts with 17 introspective practitioners that had a wide range of attitudes to both introspection and technology. This research made clear that currently available digital products fall short of supporting their needs. Our findings show that proposing concepts that exhibit potentially controversial design qualities can be productive in getting a better grasp on the design space. Through investigating concepts that appeared contentious, we were able to develop a sensibility for understanding where people’s boundaries exist, what design qualities ‘cross’ them, and how people have different viewpoints on social acceptability. Our point to the need to reveal patterns while balancing confrontation, better balance curational control and autonomy, and preserve temporal connections among a past, present, and future self.

Related Publications
for the Workshop Data as a Material for Design: Alternative Narratives, Divergent Pathways, and Future Directions @ ACM CHI 2023

Data and Generative AI as a Design Material: Co-creating abstract, irrational, and inspirational data representations for the home


Acknowledgments

  • This research is supported by the the Social Sciences and Humanities Research Council of Canada (SSHRC), Natural Sciences and Engineering Research Council of Canada (NSERC), and Canada Foundation for Innovation (CFI).

  • We thank our participants for generously sharing their experiences with us and Amir Ali, Chiara Schmitt, Chiara Ferrari, Julian Goto, and Vanessa Montoya for their assistance on this project. 

Next Project

Check out Capra