Methodology
Research Design and Methods
(Baytaş, 2020) categorizes design research into three modes: (1) conducting research to inform design decisions, (2)studying designs to generate knowledge, and (3) using design itself as a means of inquiry. My research is of the 1st category, aiming make better design design decisions for my sustainability-focused financial app. When developing my research design, I relied on the advice of (Christian Rohrer, 2022) to decide when to choose which user experience research methods.
Overview of research methods.
| Group | Task |
|---|---|
| Experts (Finance) | Interview |
| Experts (Design) | Interview |
| Experts (Sustainability) | Interview |
| Target Audience (College Students) | Survey |
| Target Audience (College Students) | Prototype Testing |
This mixed-method research design is divided into three stages.
Phase One - Qualitative Research
My purpose for the first qualitative stage is to explore the general themes arising from the literature review related to the design of AI advisors for investing. I identifed specific user experience factors, through interviewing experts in financial technology and user experience design and reviewing existing applications on the marketplace. At this stage in the research, the central concept being studied was defined generally as expectations towards a sustainable investment AI advisor.
I started with literature review, which led into expert interviews (there were many questions arising from the literature). I identified key concepts from expert discussions and gained exposure to their industry insights.
The qualitative research methods employed in the first stage of the research design enables me to explore concepts arising for literature review further, using a more open approach, without limiting the conversation only to pre-ascribed notions. The strength of the qualitative approach in the first stage is to encourage the discovery of new ideas, not yet common in literature and potential user experience factors related to sustainable investing and user experience.
Sampling
My qualitative sampling structure uses non-probability snowball sampling, with the following criteria: financial industry, fintech, and design experts; located everywhere.
Methods
Semi-Structured Interviews
I conducted exploratory research in English using semi-structured interviews recorded online and offline. Talked to 5 experts over video call and 1 one expert face-to-face; all interviews were in English. I recorded audio and video, transcribed the conversations, and then perform thematic analysis of the contents.
- Qualitative research targeted at financial, design, and sustainability experts, leading to a wish list of features.
Conceptual Framework
The conceptual framework map presents the key concepts arising from the literature review thus far in the research process. I’m using these concepts when developing interview strategies for phase one of the research, developing the survey questionnaire for phase two, as well as for building the Green Filter AI Companion for young adults at the final stage of the process. However, I expect the conceptual framework to further evolve with additional findings while conducting my research.
Conceptual Model
Phase Two - Quantitative Research
I then proceeded to the second, quantitative stage, informed by the previously identified factors, and prepared a survey, including a Likert scale, and a choice experiment, focusing on the preferences of the potential users in generation-z aged 18-29, living in Taiwan, exploring the relationship between independent variables: - “Interest in Sustainability” - “Interest in Investing” - “Preferred Features” - and the dependent variable “User Sign-ups”.
The strength of quantitative research is to enable me to access a larger sample of potential users in two countries, using online survey methods, and to validate some of the qualitative findings from stage one.
- Survey potential users’ preferences, including a choice experiment of the proposed features.
Sampling
My quantitative sampling structure uses a judgmental criterion: adults aged in Generation-Z (18-29), located in Taiwan, surveyed using a Chinese-language online survey.
Methods
Likert: The survey includes a Likert scale between 1 to 5 to validate key findings from the first stage of the research by assessing responses to statements regarding the app’s design, features, and other criteria that may still emerge.Choice Experiment: The survey includes a choice experiment between different sets of potential features available when communicating with the sustainable finance AI companion.
Data Collection
For my survey, I adopted a face-to-face method to increase response rates distributing flyers to students on college campuses, canteens, and classrooms, getting verbal permission from educators in their classrooms to distribute the survey flyer. Similarly to the approach take by (Liu et al., 2023), I distributed the survey flyer at universities located in the Northern, Soutern, Central, and East regions of Taiwan. The flyer included a colorful AI-generated visual with a futuristic game-like female figure, and the title “climate anxiety survey” in Chinese, as well as a website link (ziran.tw) and scannable QR-code.
The survey only included questions and descriptions in Chinese. I have used the Claude 3 Opus model to translate them to English for this table.
| Original Question in Chinese | English Translation |
|---|---|
| 如果你/妳懷疑你/妳要買的番茄可能是由強迫勞工（現代奴隸）採摘的，你/妳仍然會買它嗎？ | If you suspect that the tomatoes you are going to buy may have been picked by forced labor (modern slaves), would you still buy them? |
| 你/妳關心食安嗎？ | Do you care about food safety? |
| 你/妳7年內買車嗎？🚘 | Will you buy a car within 7 years? 🚘 |
| 你/妳7年內買房嗎？🏡 | Will you buy a house within 7 years? 🏡 |
| 你/妳購物時知道產品環保嗎？ | Do you know if the products are environmentally friendly when you shop? |
| 你/妳覺得認證環保的公司更好嗎？ | Do you think companies certified as environmentally friendly are better? |
| 你/妳支持肉稅嗎？ | Do you support a meat tax? |
| 你/妳關心食用雞的生活嗎？ | Do you care about the lives of chickens raised for food? |
| 你/妳避免吃肉嗎？ | Do you avoid eating meat? |
| 你/妳覺得你/妳花錢會影響環境嗎？ | Do you think your spending affects the environment? |
| 你/妳會對金錢感到焦慮嗎？ | Do you feel anxious about money? |
| 你/妳會對金錢很節儉嗎？ | Are you very frugal with money? |
| 你/妳會經常存錢嗎？ | Do you often save money? |
| 你/妳對自己的財務知識滿意嗎？ | Are you satisfied with your financial knowledge? |
| 你/妳投資會考慮環保嗎？ | Do you consider environmental protection when investing? |
| 你/妳覺得台灣的經濟目標是增長嗎？ | Do you think Taiwan’s economic goal is growth? |
| 你/妳覺台灣的得環境退化是台灣的經濟增長的前提嗎？ | Do you think environmental degradation in Taiwan is a prerequisite for Taiwan’s economic growth? |
| 你/妳覺得台灣的經濟增長有助於保護環境嗎？ | Do you think Taiwan’s economic growth helps protect the environment? |
| 你/妳覺得經濟能不排CO₂eq也增長嗎？ | Do you think the economy can grow without emitting CO₂eq? |
| 你/妳覺得經濟增長有物質限制嗎？ | Do you think there are material limits to economic growth? |
| 你/妳會每天都用AI嗎？ | Do you use AI every day? |
| 你/妳會信任AI嗎？ | Do you trust AI? |
| 你/妳想要AI有個造型嗎？ | Do you want AI to have a specific appearance? |
| 你/妳喜歡待在大自然嗎？ | Do you like being in nature? |
| 你/妳擔心氣候變化嗎？ | Are you worried about climate change? |
| 你/妳對環境污染情況會感到焦慮嗎？ | Do you feel anxious about environmental pollution? |
| 你/妳知道許多植物和動物的名字嗎？ | Do you know the names of many plants and animals? |
| 你/妳感覺自己和大自然很接近嗎？ | Do you feel close to nature? |
| 你/妳努力實踐低碳生活嗎？ | Do you strive to live a low-carbon lifestyle? |
| 你/妳想做更多環保事嗎？ | Do you want to do more for environmental protection? |
| 你/妳對環境相關政治議題有興趣嗎？ | Are you interested in environmental political issues? |
| 你/妳信任碳排放抵消額度嗎？ | Do you trust carbon offset credits? |
| 你/妳的環保行動對環境保護有效果嗎？ | Do your environmental actions have an effect on environmental protection? |
| 你/妳想在行業內推環保嗎？ | Do you want to promote environmental protection within your industry? |
| 你/妳得自己對新觀念開放嗎？ | Are you open to new ideas? |
| 你/妳的大學對可環保性支持嗎？ | Does your university support environmental sustainability? |
Respondents who remained outside the survey parameters were allowed to answer the survey however their responses were disregarded from the data analysis. References were stored in the Zotero paid version with 6 GB storage. Bibtex and Better Bibtex were used to export the references to the .bib format consumable by the Quarto scientific writing system.
Phase Three - Qualitative Research
In the third stage, I return to the qualitative methods, by building a prototype of the sustainable investing AI companion, taking into account insights gathered in the previous stage. I will used face-to-face interview to discuss the prototype, and conducted a thematic analysis of the discussions’ recordings, leading to further validation of previously gathered data and changes in the prototype. The gained insights, accompanied by the app prototype, which embodies my findings, are the final outcome of my research.
In the last phase, I will return to the qualitative methods to further validate the quantitative findings from stage two. Here my focus will be on operationalizing the gathered insights into a prototype that users can experiment with and discuss with their peers in a focus group setting.
- Design a prototype of the personal sustainable finance AI assistant, tested using qualitative methods in a focus group.
Sampling
The phase three sampling structure uses a judgmental criterion: - age Gen-Z cohort - located in Taiwan - Using Chinese for discussion.
Individual face-to-face interviews were organized at universities around Taiwan. Because in-person presence is required in this stage, the prototype will only be tested by potential users physically present in Taiwan. To avoid convenience sampling, I posted online ads in Chinese and reach out to varied student clubs to invite people who I don’t know personally, to participate in a “sustainable AI application testing” (wording may change).
Methods
Face-to-face prototype testing. The strength of in-person is the ability to observe potential users, where knowledge can be exchanged directly. The interviews were recorded and transcribed.
Thematic Analysis Finally, I will perform a thematic analysis of the focus group transcriptions in order to validate previous findings, and open avenues for future research.
Interview transcripts from Descript and Google Speech-to-Text model were combined using Gemini 2.5 Pro Experimental 03-25 (gemini-2.5-pro-exp-03-25) model.
Prototype Testing
Does the prototype match user needs?
Testing was the most difficult part of the thesis process.
-
Testing with random people found at the university.
-
Testing with experts.
-
Testing with unknown people in the target audience.
App Testing Flow
1st Wave of (Preliminary) Prototype Testing (Spring 2024)
The 1st wave of preliminary testing took place during 1 month from 2024 April 2 to May 2, 2024 at the NCKU campus. I found 8 anonymous participants at different NCKU locations, such as the student canteens and the medical library, confirmed the people I approached were Taiwanese students studying at NCKU, and then simply asked them to test my app, using my own laptop. I made used of participant observation and took notes myself. There was no audio or video recording.
2nd Wave of (In Production) Prototype Testing (Autumn 2024 - Spring 2025)
The 2nd wave of testing took place from Autumn 2024 to Spring 2025 and was more comprehensive. I conducted in-person face-to-face testing individually with 32 students at 7 universities. Interviews were conducted in Chinese and transcribed and translated to English using Google Voice to Text AI as well as Describe AI.
| Region | University | No of Testees |
| Taichung | 國立中興大學 National Chung Hsing University (NCHU) | 7 |
| Chiayi | 國立中正大學 National Chung Cheng University (CCU) | 5 |
| Tainan | 國立成功大學 National Cheng Kung University (NCKU) | 6 |
| Tainan | 國立臺南藝術大學 Tainan National University of the Arts (TNNUA) | 2 |
| Tainan | 長榮大學 Chang Jung Christian University (CJCU) | 5 |
| Tainan | 南臺科技大學 Southern Taiwan University of Science and Technology (STUST) | 5 |
| Pingtung | 國立屏東科技大學 National Pingtung University of Science and Technology (NPUST) | 2 |