Skip to main content

Methodology

Start with expert survey, then expert interviews (because so many questions from literature review).

Group Task
Experts (Finance) Interview
Experts (Design) Interview
Experts (Sustainability) Interview
Target Audience (College Students) Survey + Prototype Testing

I adopted a face-to-face method to increase response rates distributing flyers to students on college campuses, canteens, and classrooms getting verbal permission from educators in their ´ classrooms to distribute the survey flyer. The flyer included a colorful AI-generated visual with a futuristic game-link female figure, the title “climate anxiety survey”, a website link (ziran.tw) and scannable QR-code.

Similarly to (Liu et al., 2023) I distributed the survey in schools in the Northern, Soutern, Central, and East regions of Taiwan.

The survey only included questions and descriptions in Chinese. I have used the Claude 3 Opus model to translate them to English for this table.

36 Likert Fields included in the survey
Original Question in Chinese English Translation
如果你/妳懷疑你/妳要買的番茄可能是由強迫勞工（現代奴隸）採摘的，你/妳仍然會買它嗎？ If you suspect that the tomatoes you are going to buy may have been picked by forced labor (modern slaves), would you still buy them?
你/妳關心食安嗎？ Do you care about food safety?
你/妳7年內買車嗎？🚘 Will you buy a car within 7 years? 🚘
你/妳7年內買房嗎？🏡 Will you buy a house within 7 years? 🏡
你/妳購物時知道產品環保嗎？ Do you know if the products are environmentally friendly when you shop?
你/妳覺得認證環保的公司更好嗎？ Do you think companies certified as environmentally friendly are better?
你/妳支持肉稅嗎？ Do you support a meat tax?
你/妳關心食用雞的生活嗎？ Do you care about the lives of chickens raised for food?
你/妳避免吃肉嗎？ Do you avoid eating meat?
你/妳覺得你/妳花錢會影響環境嗎？ Do you think your spending affects the environment?
你/妳會對金錢感到焦慮嗎？ Do you feel anxious about money?
你/妳會對金錢很節儉嗎？ Are you very frugal with money?
你/妳會經常存錢嗎？ Do you often save money?
你/妳對自己的財務知識滿意嗎？ Are you satisfied with your financial knowledge?
你/妳投資會考慮環保嗎？ Do you consider environmental protection when investing?
你/妳覺得台灣的經濟目標是增長嗎？ Do you think Taiwan’s economic goal is growth?
你/妳覺台灣的得環境退化是台灣的經濟增長的前提嗎？ Do you think environmental degradation in Taiwan is a prerequisite for Taiwan’s economic growth?
你/妳覺得台灣的經濟增長有助於保護環境嗎？ Do you think Taiwan’s economic growth helps protect the environment?
你/妳覺得經濟能不排CO2也增長嗎？ Do you think the economy can grow without emitting CO2?
你/妳覺得經濟增長有物質限制嗎？ Do you think there are material limits to economic growth?
你/妳會每天都用AI嗎？ Do you use AI every day?
你/妳會信任AI嗎？ Do you trust AI?
你/妳想要AI有個造型嗎？ Do you want AI to have a specific appearance?
你/妳喜歡待在大自然嗎？ Do you like being in nature?
你/妳擔心氣候變化嗎？ Are you worried about climate change?
你/妳對環境污染情況會感到焦慮嗎？ Do you feel anxious about environmental pollution?
你/妳知道許多植物和動物的名字嗎？ Do you know the names of many plants and animals?
你/妳感覺自己和大自然很接近嗎？ Do you feel close to nature?
你/妳努力實踐低碳生活嗎？ Do you strive to live a low-carbon lifestyle?
你/妳想做更多環保事嗎？ Do you want to do more for environmental protection?
你/妳對環境相關政治議題有興趣嗎？ Are you interested in environmental political issues?
你/妳信任碳排放抵消額度嗎？ Do you trust carbon offset credits?
你/妳的環保行動對環境保護有效果嗎？ Do your environmental actions have an effect on environmental protection?
你/妳想在行業內推環保嗎？ Do you want to promote environmental protection within your industry?
你/妳得自己對新觀念開放嗎？ Are you open to new ideas?
你/妳的大學對可環保性支持嗎？ Does your university support environmental sustainability?

Respondents who remained outside the survey parameters were disregarded from the data analysis.

  • References were stored in the Zotero paid version with 6 GB storage.
  • Bibtex and Better Bibtex were used to export the references to the .bib format consumable by Quarto.

Research Design​

When developing my research design, I relied on the advice of (Christian Rohrer, 2022) to decide when to choose which User-Experience Research methods.s

The research design of this study consists of 3 steps.

  • Qualitative research targeted at financial and interaction design experts, leading to a wish list of features.
  • Survey potential users’ preferences, including a choice experiment of the proposed features.
  • Design a prototype of the personal sustainable finance AI assistant, tested using qualitative methods in a focus group.
  • This mixed-method research design is divided into three stages (fig. 11).
  • My purpose for the first qualitative stage is to explore the general themes arising from the literature review related to the design of AI advisors for investing. I will identify specific user experience factors, through interviewing experts in financial technology and user experience design and reviewing existing applications on the marketplace. At this stage in the research, the central concept being studied is defined generally as expectations towards a sustainable investment AI advisor.
  • I will then proceed to the second, quantitative stage, informed by the previously identified factors, and prepare a survey, including a Likert scale, and a choice experiment, focusing on the preferences of the potential users in generation-z aged 18-29, living in Taiwan (see fig. 12 on next page), exploring the relationship between independent variables: - “Interest in Sustainability” - “Interest in Investing” - “Preferred Features” - and the dependent variable “User Sign-ups”.
  • In the third stage, I will return to the qualitative methods, by building a prototype of the sustainable investing AI companion, taking into account insights gathered in the previous stage. I will use a focus group to discuss the prototype, and conduct a thematic analysis of the discussions’ recordings, leading to further validation of previously gathered data and possible changes in the prototype. The gained insights, accompanied by the app prototype, which embodies my findings, will be the final outcome of my research.

Conceptual Framework​

  • The conceptual framework map presents the key concepts arising from the literature review thus far in the research process. I’m using these concepts when developing interview strategies for phase one of the research, developing the survey questionnaire for phase two, as well as for building the Personal Sustainable AI Financial Advisor (PSAA) for young adults at the final stage of the process. However, I expect the conceptual framework to further evolve with additional findings while conducting my research.
Concept map.

Conceptual Model

Research Methods​

Phase One - Qualitative Research​

The qualitative research methods employed in the first stage of the research design enables me to explore concepts arising for literature review further, using a more open approach, without limiting the conversation only to pre-ascribed notions. The strength of the qualitative approach in the first stage is to encourage the discovery of new ideas, not yet common in literature and potential user experience factors related to sustainable investing and user experience.

Sampling​

My qualitative sampling structure uses non-probability snowball sampling, with the following criteria: financial industry, fintech, and design experts everywhere, including in Taiwan and Sweden, but also Estonia, Portugal, and elsewhere.

Method: Semi-Structured Interviews

I will conduct exploratory research in English using semi-structured interviews recorded online and offline.

Phase Two - Quantitative Research​

The strength of quantitative research is to enable me to access a larger sample of potential users in two countries, using online survey methods, and to validate some of the qualitative findings from stage one.

Sampling My quantitative sampling structure uses a judgmental criterion: adults aged in Generation-Z (18-29), located in Taiwan, surveyed using a Chinese-language online survey.

Method: Likert The survey includes a Likert scale between 1 to 5 to validate key findings from the first stage of the research by assessing responses to statements regarding the app’s design, features, and other criteria that may still emerge.

Method: Choice Experiment The survey includes a choice experiment between different sets of potential features available when communicating with the sustainable finance AI companion.

Phase Three - Qualitative Research​

In the last phase, I will return to the qualitative methods to further validate the quantitative findings from stage two. Here my focus will be on operationalizing the gathered insights into a prototype that users can experiment with and discuss with their peers in a focus group setting. Sampling

The phase three sampling structure uses a judgmental criterion: - age 18-35 - located in Taiwan - Using the Chinese language for discussion

Individual face-to-face interviews with be organized at universities around Taiwan. Because in-person presence is required in this stage, the prototype will only be tested by potential users physically present in Taiwan. To avoid convenience sampling, I will post online ads in Chinese and reach out to varied student clubs to invite people who I don’t know personally, to participate in a “sustainable AI application testing” (wording may change).

Method: Focus Group The strength of a focus group is the ability to observe potential users in a social setting, where knowledge can be exchanged between the participants. The whole experience, including emerging conversations, will be recorded and transcribed.

Method: Thematic Analysis Finally, I will perform a thematic analysis of the focus group transcriptions in order to validate previous findings, and open avenues for future research.

Overview of research methodology.
  • Interview transcript from Descript and Google Speech-to-Text model were combined using Gemini 2.5 Pro Experimental 03-25 (gemini-2.5-pro-exp-03-25) model.

Testing​

Does the prototype match user needs?

Testing was the most difficult part of the thesis process.

  • Testing with friends and acquaintances.

  • Testing with experts.

  • Testing with unknown people in the target audience.

App Testing Flow​

1st Wave of Testing (Spring 2024)​

  • 1 month from 2024 April 2 to May 2, 2024
  • 8 anonymous participants at NCKU
  • Participant observation
  • No audio recording

Prototype Testing results

sun, 14. april. 22h at D24

note: there’s dropoff on evey step of the user journey

RQ: To what extend can shopping become and entry point for saving and investing. RQ: Can shopping serve as an entry point for sustainable saving and investing?

Testing overview

  • Tested with 30 participants individually
  • Testing is anonymous
  • Generate

有意識的消費主義

  • add carbon indicators, other labels to the analsysi, add report code, calculate report code from URL? ssave as kv
may 8 czuta
- investment help useless.. needs simpler intro - wants to see real cows - very curious abouut companes - wants to see profit percentage. why is margin so high if polluution is bad. - real environmental impact of the coompany.

06 may - Seeing factory photos is useful only if they are trustworthy photos. Who will provide them? -

may 5
- user: it looks like an ad
may 3
- uses google to look for “fashion brand eco friendly”
- thinks “goodonyou.eco” looks like a brand website
may3 - first looked for NET clothes but Momo doesn’t sell it - Looked for Sony camera lens

1 may prof suggestion - make connection between biodibersity and production and consumption clearer - what is the incentive for companies to share their data?

my own idea: like the switch of going from traditional banking with ATM machines on the street (or even the physica bank office) to online banking with mobile payments

  • hypothesis: esg accessibility can push companies to increase production standards

  • what if you can see ESG in near-realtime such as the stock market price

  • i can imagine esg derivative product like siemens gamesa

  • ai can help integrate esg derivatives into daily life to drive esg adoption

  • “effective altruism (EA)”

  • “Blockchain technology can improve price transparency in product distribution by allowing consumers to know the exact pricing from raw materials to distributors to suppliers.”

Interviews and testing survey were conducted anonymously in hopes to have more honest responses from the responders.

  • ziran chrome extension is unable to record activity due to browser security restrictions for plugins
  • ai.ziran.tw record user activity
  • semi-structured interviews were conducted in chinese
  • the interviewer (me) took notes of the interviews
  • some gaps in the data exist due to the limited chinese language skills of the interviewer (me)

basic interview script - momo: what is a brand that you like’d or would like to buy - search - please pick a product (or search again) - on product page: what do you notice on this page? - what kind of information is important for you on this page? - do you notice anything else - (if the user doesn’t notice the green filter, direct their attention to it and ask: what do you think this does?) - would you click on it - if the user says yes, continue - if thhe user says no, make note and continue - as the extension generates a response: what do you think about this content? - is there any information that you consider important? - anything else you see that you think looks special - do you see anywhere you can click? - would you click on it? - if yes, continue - if no, make note, and continue - explain: due to the limitation of the prototype, the test will continue on a separate page where you can ask questions - is there anything you would like to ask the helper? - notice if the user picks from sample questions - remind the user they can come up with their own question - as the ai is generating content ask: do you see any information in this content - did you know this before or is there any info you didn’t know before? make note. - front page: explain the helper takes into account your personal info and goals. - ask: what kind of information do you think important to share with the helper?

Tuesday 30. April 14:05-14:45 - 7CYQ6

  • Momo
  • Looks for Levis pants
  • Looks for recommendations on the sidebar
  • Looks at the photos
  • Looks at the price and options
  • Didn’t notice the helper as it looks like an ad
  • When helped
  • Ignores 社區支持： 購物 69% 儲蓄 80% 投資 65% as doesn’t know what these mean
  • on ai.ziran
  • shares personal info: 四年後想考研究所，還不想工作，所以不會存到錢，希望可以考到台北的學校，每個月有兩萬生活費。
  • DJmoney
  • https://www.moneydj.com/etf/x/basic/basic0004.xdjhtm?etfid=0050.tw
  • Still didn’t notice the helper
  • Doesn’t understand investing (Understands it’s Taiwanese stocks) so the helper is useful for explaining new concepts

Monday 29. april 10:10-10.25 - Momo - buy new balance sneakers

  • DJmoney
  • wants compare EFTs

Monday 29. april 14:50-15:10 - Momo - buy apple iphone - bad internet - app was slow - app crashed

  • green filter analysis on DJmoney seems more trustworthy than other 2 eft sites
  • button placement is important (too low on other sites than djmoney)

Sunday 28. april 16:00 - momo: wants to buy ice cream

sun, 14. april. 22h at D24

note: there’s dropoff on evey step of the user journey

RQ: To what extend can shopping become and entry point for saving and investing. RQ: Can shopping serve as an entry point for sustainable saving and investing?

Testing overview

  • Tested with 30 participants individually
  • Testing is anonymous
  • Generate

有意識的消費主義

  • add carbon indicators, other labels to the analsysi, add report code, calculate report code from URL? ssave as kv

Expert Feedback​

  • Provide alternatives

  • Simplify text

  • Use images

  • Put a “New Feature” ad on the front page

  • Change 繼續討論 to something more actionable

    • I tried changing to “see alternatives”

2nd Wave of Testing (Autumn 2024 - Spring 2025)​

In-Person Face-to-Face Testing

I conducted in-person testing individually with 32 students at 7 universities. Interviews were conducted in Chinese and transcribed and translated to English using Google Voice to Text AI as well as Describe AI. Face-to-face testing was concluded in several universities across Taiwan.

Region University No of Testees
Taichung 國立中興大學 National Chung Hsing University (NCHU) 7
Chiayi 國立中正大學 National Chung Cheng University (CCU) 5
Tainan 國立成功大學 National Cheng Kung University (NCKU) 6
Tainan 國立臺南藝術大學 Tainan National University of the Arts (TNNUA) 2
Tainan 長榮大學 Chang Jung Christian University (CJCU) 5
Tainan 南臺科技大學 Southern Taiwan University of Science and Technology (STUST) 5
Pingtung 國立屏東科技大學 National Pingtung University of Science and Technology (NPUST) 2
Last updated on by Kris Haamer