This three-month solo project used a mixed-methods research approach to design low-fidelity wireframes and design guidelines for a generative artificial intelligence tool.
Ten professional journalists were recruited for this study. Quantitative research was conducted using DMINR cards, a design tool with journalistic concepts and design provocations around AI. Qualitative data was collected through semi-structured interviews, and a focus group was conducted to collect user evaluations.
Role
Lead Researcher and Designer
Timeline
July 2024 - October 2024
Skills and Tools
Competitive Analysis and Desk Research
Semi-structured interviews
Canva for Flyers
Figma to design wireframes
Descript to transcribe interviews
Maze for conducting user interviews
Excel for quantitative analysis and thematic analysis.
Team
Janie Hendrickson
PHASE I: RESEARCH
Initial desk research started by subscribing to newsletters such as Shelly Palmer, watching the Walkley Foundation and Investigative Reporters & Editors (IRE) conference videos, and joining LSE's JournalismAI slack group community. A thorough literature review was also conducted with an analysis of 10 published papers.
Then, the study planning began:
1. I prepared the study materials, such as a schedule, an interview script, participant information sheets & consent forms, and set up Maze, a User Research website, for the participants to play the card games on their own devices.
2. Recruitment started in the second week of July and continued until the end of August through advertisements via social media (LinkedIn posts), flyers, emails, personal contacts, and LSE’s JournalismAI slack group that I joined.
3. I conducted a Pilot Study with two writers and not professional journalists to ensure there would be enough journalists for the actual study, and to avoid repeating participants.
4. I briefed participants by sending them the participant information sheets and consent forms to be signed.
DMINR Cards- What are they?
Generative AI (gAI) has the potential to transform journalism, but most AI tools are designed without input from journalists, leading to tools that may not align with journalistic values. The DMINR project, a collaboration between City, University of London, and the Google News Initiative, aims to include journalists in AI tool design. The project produced resources like a card deck to facilitate the design process. Two decks—focused on journalistic workflows and AI opportunities and challenges—were used in this study to help ensure an accurate representation of investigative journalists' values and workflows during interviews.
PHASE II: CONCEPT
Data Analysis:
Three layers of analysis were conducted.
Quantitative Approach:
- Used Maze functionality to analyze data, which was entered into Excel.
- Created formulas for averages, verified them against recordings, and visualized them with an agreement matrix and graphs.
Deductive (Top-Down) Approach:
- Extracted themes for RQ1 and RQ2 from transcriptions.
- Notable themes were recorded in Excel, and participants mentioned them.
Inductive (Bottom-Up) Approach:
- Supporting data for specific quotes was added in a separate Excel tab.
Key Insights:
-Per the results, journalists predominately use AI for the following tasks: doing research for a story, gathering evidence, finding sources of information, and making sense of information.
-Journalists do not use AI as much for finding a story angle, writing a story, publishing a story, exploring story leads, or looking for newsworthy stories.
-Journalists do not use AI, but would use it in the future for verifying information, exploring information or datasets, and corroborating facts.
PHASE III and IV: Design and DEVELOP
Ensure easy onboarding and a low barrier of entry (addresses Challenge #1).
-Particpiant 10's (Head of Content) team received extensive gAI training, leading to more AI exploration.
-Participant 3's (Editor-in-Chief) company developed an internal GPT tool, but she feels unprepared to use it due to a lack of training and guidance.
Key Insights:
-Accessible and user-friendly AI tools are crucial for seamless adoption, especially for journalists without formal training.
-Journalists are unlikely to adopt new tools unless the benefits are clear and immediate.
Design Priority: HCI professionals should focus on user-friendliness, as comprehensive training like P10's team received is rare.
Ensure job satisfaction remains (addresses Challenge #2).
-Many journalists prefer finding themes and story angles themselves (Participant 2, Participant 5, Participant 7).
-Participant 3 (Editor-in-Chief) hesitates to adopt AI, fearing it might reduce job satisfaction and creativity.
-Journalists want to retain creative control, with AI enhancing rather than replacing tasks.
-Participant 6 (Investigative Journalist & Professor) suggests gAI should act as a “skills amplifier,” optimizing tasks like writing and generating headlines, not doing them entirely.
-Participant 3 (Editor-in-Chief) hesitates to adopt AI, fearing it might reduce job satisfaction and creativity.
-Journalists want to retain creative control, with AI enhancing rather than replacing tasks.
-Participant 6 (Investigative Journalist & Professor) suggests gAI should act as a “skills amplifier,” optimizing tasks like writing and generating headlines, not doing them entirely.
Transparency with Sources (address Challenge #3)
-Participant 7 (Bureau Chief) emphasized ethical standards, prioritizing fact-checking over scoops.
-Participant 10 (Head of Content) wants AI to show where information comes from, avoiding fake URLs clearly.
-Participant 3 (Editor-in-Chief) seeks AI that provides background on new subjects and links to verified sources.
-Participant 7 (Bureau Chief) emphasized ethical standards, prioritizing fact-checking over scoops.
-Participant 10 (Head of Content) wants AI to show where information comes from, avoiding fake URLs clearly.
-Participant 3 (Editor-in-Chief) seeks AI that provides background on new subjects and links to verified sources.
Key Insights:
-Journalists value truth-telling and accurate attribution.
- Trust in AI tools requires control over the origin of sources, especially due to hallucination risks.
Design Priority: HCI professionals should focus on designing transparent AI tools, with a focus on source clarity and feedback mechanisms.
PHASE V: EVALUATE
I prepared six low-fidelity wireframes and a presentation showing the study objective and results to guide the conversation. After a 10-minute presentation, the researcher asked open-ended questions about the wireframes and design guidelines to get formative feedback.
Key Insights: Need for AI Training:
-Michelle emphasized the importance of AI training in journalism.
-Newsrooms, like Rihanna’s, are hiring data journalists to train AI for specific use cases.
-Time required to learn AI tools hinders adoption.
-Michelle emphasized the importance of AI training in journalism.
-Newsrooms, like Rihanna’s, are hiring data journalists to train AI for specific use cases.
-Time required to learn AI tools hinders adoption.
Human Element in Journalism:
-P1 agrees with keeping human involvement, especially in tasks like headline writing.
-She prefers AI suggestions but doesn’t want AI to fully take over creative tasks.
-P1 agrees with keeping human involvement, especially in tasks like headline writing.
-She prefers AI suggestions but doesn’t want AI to fully take over creative tasks.
Concerns About AI Fact-Checking:
-P1 worries that AI could amplify incorrect information, creating an "echo chamber."
-Discussed potential for AI to identify discrepancies rather than fact-check.
-P1 worries that AI could amplify incorrect information, creating an "echo chamber."
-Discussed potential for AI to identify discrepancies rather than fact-check.
Rapid AI Adoption:
-P1 noticed increased AI integration in tools like Google Gemini and Zoom since July.
-Urgency for more research to inform well-designed AI systems for journalists.
-P1 noticed increased AI integration in tools like Google Gemini and Zoom since July.
-Urgency for more research to inform well-designed AI systems for journalists.
CALL TO ACTION
As gAI continues to evolve, its role in journalism will expand, but it will need to do so in a way that complements journalists, such as gathering and structuring information, optimizing text, and corroborating facts. Designers of digital systems for journalists should incorporate these findings into their user research to create products that address the specific needs of journalists. The intersection between professional journalism and the design of algorithmic systems remains underexplored, so this study also offers valuable insights for both HCI researchers and journalism scholars, fostering more shared knowledge in this emerging space.
