2023/8

Improved research article search accuracy by integrating an AI agent for keyword refinement.

Improved research article search accuracy by integrating an AI agent for keyword refinement.

As a UXUI designer, I helped shape the AI-assisted literature search experience at Transformative Learning Technologies Lab ↗. I identified key pain points in literature hunt through user research, designed AI-assisted keyword generation and optimization flows to improve search efficiency and recommendation relevance, and built a design system with unified typography, color, and component standards to support consistency and faster team cooperation.

Product

AI-powered Literature Search Tool

Team

Yolanda (me), UXUI designer

Dora, UXUI designer

Peiying, UX designer

Sabrina, UX designer

Yipu Zheng, product lead & engineer

Timeline

8 weeks (design sprint)

Skills

Design Sprint · User Research · UI design guidelines · Interactive prototyping

Results

The cornerstone for

Curiously won the Catalyst Prize at the 2024 Learning Engineering Tools Competition in 2024 by Yipu Zheng.

SOLUTIONS HIGHLIGHT

STEP 1 SELECT

Here are the keywords you can use.

Use academic terms over self-created ones for precision.

  • Instant Feedback: clicking on the keywords refreshes the results on the right automatically

  • Categorized Keywords: keywords are grouped by categories, content, methodology, etc. 

  • Selection Recap: you can find what keywords selected previously to generate the results

STEP 2 REGENERATE

Don't worry, try again.

It is normal that a one-time search is not enough.​Instant Feedback: clicking on the keywords refreshes the results on the right automatically

  • Regenerate Button: click the "Regenerate" button to let AILit refresh itself and do the search again

  • Search History: you can find first-round keywords in the history by clicking on the pagination

STEP 3 EDIT MANUALLY

Don't worry, try again.

It allows manual editing to happen more seamlessly.

  • Edit and Save: users can manually replace selected keywords with other edited keywords

  • Sync and Search: the newly edited keywords are synced on the reminder on the chatbot automatically

BACKGROUND

About

The Transformative Learning Technologies Lab is a STEM education research center at Teachers College, Columbia University. One of the lab’s core missions is to explore emerging technologies and think critically about how they can be applied in educational settings to improve learning.

Why this Project?

At the end of 2022, ChatGPT was released. This project was driven by a simple goal: to identify a problem rooted in students’ everyday lives and explore how this new technology, ChatGPT, or similar AI Chatbot agent, could be used to improve educational outcomes and performance.

Resources

In March 2023, my boss Yipu Zheng published a paper titled “Outside-in, Inside-out, or Outside-out? Exploring the Flow of New Ideas and the Development of CSCL and ICLS via Co-Authorship Networks from 1995 to 2020.” The study analyzed the field of Learning Sciences through keyword analysis and social network analysis, covering all related papers published between 1995 and 2020.


The research produced a large and valuable dataset, including 5,091 papers and 8,225 authors. This became the key resource for our project. Building on this database, we aimed to design a new tool or experience that could translate complex academic research into something more meaningful, accessible, and useful in an educational context.

It's time to think of product ideas.

BRAINSTORM

Our first brainstorm session

Doing literature search is a pain

During the first team meeting, we brainstormed about the product ideas. In what scenarios can a ChatGPT-like AI Agent can help students better? We land on the literature research experience itself.


Every college student has to do literature research. And we all know. It is a real pain.


Can ChatGPT help with this? At the beginning of 2023, ChatGPT had limitations. It can only give fake links and sometimes make up research findings.

Questionnaire & Interview & Secondary Research

After brainstorming, we together listed out the specific questions we would like to dig deeper into.

-> Questionnaire

to understand students' research behavior

to know more about their habits in using AI tools

-> Interview

to find reasons, pain points, and problems

to talk about their wild imaginations

-> Research

to understand the strengths and limitations of AI tools

to know the theory behind "how to ask a good research questions"

Ok. So what are the findings?

USER RESEARCH

Questionnaire Results

Keywords VS Question

100% students in our samples use keywords during the initial paper hunting process.

They care about impact

Students prefer big titles, researchers, big numbers of citations.

Results in a Page

Counterintuitively, more than half of the students prefer 10~20 results returned in one page. No less than 10, no more then 20.

AI's Role

During the research and interviews, we concluded that AI Agent perform better when they are in an assisted role during the initial search phase.

Time to map out the userflow and do the wireframes.

WIREFRAMES

User flow and decision points

We used mindmap to map the core research journey, including how users enter search intent, receive AI-generated keyword suggestions, refine results, and access recommended papers. This helped define the key steps, reduce friction, and identify where users might need guidance.

Mindmap Brainstorms

Early validation

The wireframes served as a tool for discussion with teammates and stakeholders, allowing us to align on layout, interactions, and product direction before moving into higher-fidelity design.

Wireframes

Finally, the results.

THE WHOLE PICTURE

THE UI KIT

USER RESEARCH

Users sometimes don't understand themselves

During the usability testing, I asked about which part you liked the most. Most of the users said they prefer the colorful badges. But it turns out that the badges only look pretty but they do not make sense logically. As a designer, I have to have my own critical judgment over the user feedback.

THEORETICAL BACKUP

Theory complements the interview

Our team reviewed a large number of papers. Through this secondary research process, we built a strong understanding of the topic early on, which reduced the need for more exploratory research later.