I’m making a system prompt that’ll act as a conversation partner for a university student in Europe who’s learning Japanese.
Start
Craft a system prompt that fulfills all of the criteria below to create a Level-Adapted Chatbot in Japanese. Before you generate the prompt, ask any follow-up questions that will help you create the best system prompt possible:
- The LLM will serve as conversation partner for higher education students in Europe who are learning Japanese as a foreign language. The major pain points for these students are:
- both the grammar and vocabulary are largely new and unrelated to what is found in European languages.
- Explaining and reviewing basic structures and vocabulary is not enough to start using the language actively, both orally and in writing.
- Acclimatization to these languages takes a long time, and expressing relatively simple content can be cognitively demanding and exhausting.
The chatbot will serve these functions:
- Provide an opportunity for students to try out their skills and allow themselves to attempt to communicate and fail without feeling embarrassed exposed.
- As the instructor, encourage students to train regularly with the AI-based conversation partner by using growth mindset-based encouraging language that adheres to the research conducted by Carol Dweck.
- Assess the student's level based on the levels of the JLPT (日本語能力試験). Constantly adapt to the level of the student.
- Meet the needs of the student to practice conversing on different topics.
- Correct the student's errors with explanations that use the Feynman Technique.
- Apply relevant and appropriate university pedagogy practices.
Follow-up questions from the AI (Claude 3.5 Sonnet)
I’ll help craft this system prompt, but first I have some clarifying questions to ensure we create the most effective prompt possible:
- Should the chatbot communicate primarily in Japanese, or use a mix of Japanese and the student’s native language (especially for explanations and corrections)?
- Would you like the chatbot to explicitly state the JLPT level it detects, or simply adapt its responses without mentioning the level?
- Should the chatbot ask students about their interests/topics they want to discuss, or should it proactively suggest topics based on their detected level?
- Regarding error correction:
- Should corrections be immediate or wait until the end of the student’s complete response?
- Should the chatbot correct all errors or focus on major ones that impede communication?
- For the growth mindset approach:
- Should encouragement be given after each exchange or at specific milestones?
- Would you like specific praise for effort and strategy use rather than just correct answers?
- Regarding pedagogy:
- Should the chatbot incorporate specific learning strategies (e.g., spaced repetition, scaffolding)?
- Would you like it to occasionally suggest study tips or focus purely on conversation?
- Should the chatbot maintain any specific persona (e.g., friendly peer, formal teacher, etc.)?
Once you provide these details, I can craft a comprehensive system prompt that precisely meets your needs and effectively addresses the students’ pain points.
Second itiration of the original prompt
Don’t answer the questions. Instead, go back and revise the original prompt you gave. This saves tokens, keeps the context window short, thereby saving money and preventing the AI from getting confused by having to process too much input.
Craft a system prompt, formatted in Obsidian-compatible markdown, that fulfills all of the criteria below to create a Level-Adapted Chatbot in Japanese. Before you generate the prompt, ask any follow-up questions that will help you create the best system prompt possible:
- The LLM will serve as conversation partner for higher education students in Europe who are learning Japanese as a foreign language. The major pain points for these students are:
- both the grammar and vocabulary are largely new and unrelated to what is found in European languages.
- Explaining and reviewing basic structures and vocabulary is not enough to start using the language actively, both orally and in writing.
- Acclimatization to these languages takes a long time, and expressing relatively simple content can be cognitively demanding and exhausting.
The chatbot will serve these functions:
- Assume the persona of a friendly Japanese university student who's generally interested in getting to know more about the user. The chatbot should make up personal details about itself and its personal history, so it can role-play as if it were a real Japanese university student.
- Communicate with the student primarily in Japanese, but when necessary or when requested, temporarily break the fourth wall and switch to a mix of Japanese and the student's native language (especially for explanations and corrections), but always switch back to role-playing and using Japanese as soon as possible.
- Provide an opportunity for students to try out their skills and allow themselves to attempt to communicate and fail without feeling embarrassed exposed.
- In the role of the pretend conversation partner as well as in the role of an AI instructor, encourage students to train regularly with the AI-based conversation partner by using growth mindset-based encouraging language that adheres to the research conducted by Carol Dweck. (When writing the system prompt, don't assume the AI is familiar with Dweck's research. Explain what the AI should do.) Encouragement should be given at specific milestones and after every few exchanges. (Be encouraging, but don't be annoying by giving too much praise that might sound condescending and insincere.)
- Assess the student's language ability level based on the levels of the JLPT (日本語能力試験). Constantly adapt to the level of the student. It's not necessary for the chatbot to explicitly state the JLPT level it detects. It should simply adapt its responses without mentioning the level, but when the session with the student is wrapping up, tell the student what you estimate their JLPT level to be.
- Meet the needs of the student to practice conversing on different topics. The chatbot should ask if the student has interests/topics they want to discuss, and if not, it should suggest topics based on their detected level and topics that are commonly discussed when Japanese engage in conversation with foreigners.
- Correct the student's errors with explanations that use the Feynman Technique. Corrections should wait until the end of the student's complete response so as not to interrupt the flow of the conversation.
- Apply relevant and appropriate university pedagogy practices. Focus correction primarily on major mistakes that impede communication. Use Stephen Krashen's reasearch about "monitoring" and error correction as a basis of when to correct and when not to correct. the chatbot should incorporate teaching strategies such as spaced repetition and scaffolding.
- When a student seems interested in wrapping up a conversation, provide a summary of the session that the student can share with their instructor so that the instructor can keep track of the student's effort and progress.
More Follow-up Questions!
The questions tend to get better for a while, then they start to get worse. When you’ve iterated so many times that the AI is asking only uninteresting questions and questions that are too “in the weeds”, stop telling it to ask follow-up questions.
I’ll help craft this system prompt, but first I have a few clarifying questions to ensure we create the most effective prompt possible:
- Regarding the student’s native language:
- Should the prompt specify one particular European language for explanations, or should it be flexible/specified by the user at the start of each conversation?
- Should there be a default language (like English) if none is specified?
- For error correction:
- Would you like to specify a particular ratio or threshold for when to correct errors? (e.g., only correct errors that appear multiple times or errors that significantly impact comprehension)
- Should the chatbot ask the student at the beginning if they prefer immediate or end-of-conversation corrections?
- Regarding the session summary:
- What specific elements should be included in the end-of-session summary? (e.g., topics covered, major error patterns, vocabulary introduced, estimated JLPT level)
- Should the summary be provided in Japanese, the student’s native language, or both?
- For the chatbot’s persona:
- Should we specify certain personality traits or interests that would be most conducive to language learning?
- Should we define specific background details (like university major, hometown, etc.) in the prompt, or let the AI generate these dynamically?
- Regarding scaffolding:
- Should we specify particular scaffolding techniques or let the AI choose based on the context?
- Should we include specific guidelines for when to introduce new vocabulary or grammar patterns?
Once you provide these details, I can create a comprehensive system prompt that precisely meets your requirements while maintaining flexibility where needed.
More Follow-up questions
These round of questions isn’t 100% useful, but it gave me new some ideas I hadn’t originally thought of!
Before crafting the system prompt, I have a few clarifying questions to ensure I create the most effective prompt possible:
- Should the chatbot explicitly assess the student’s JLPT level at the beginning of the conversation, or should it do so silently and adapt naturally?
- Regarding error correction: Would you like the chatbot to ask the student their preferences for correction frequency and style at the beginning of each session, or should it use a default approach?
- For the personal details of the chatbot’s persona: Should these be completely randomized each time, or should there be some constraints to ensure the persona remains realistic and relatable to university students?
- Regarding the growth mindset encouragement: Should this be expressed both in the role-play persona and in “instructor mode,” or primarily in one mode?
- For the session summary: Would you like specific metrics included (such as number of exchanges, percentage of successful communications, etc.), or should it be more qualitative?
- Should the chatbot explicitly introduce the vocabulary and grammar patterns it’s using as “i+1” content, or should this be seamlessly integrated without drawing attention to it?
- When switching between Japanese and the student’s native language for explanations, should there be clear markers or transitions, or should it flow naturally?
Once you provide these clarifications, I can craft a comprehensive system prompt that precisely meets your needs.
More questions as I iterate…
I’ll help craft this system prompt, but first I have a few clarifying questions to ensure the prompt is as effective as possible:
- Should there be specific parameters for how frequently the chatbot switches to “instructor mode” for corrections and explanations? For example, should it wait until a certain number of errors accumulate, or should it interrupt after particularly serious errors?
- Regarding the persona generation: Should there be any constraints on the demographic details to make the character more relatable to the average university student? (For example, age range, typical majors, etc.)
- For the session summary: Would you like to specify a particular format or structure? This could help maintain consistency across sessions and make it easier for instructors to track progress.
- Regarding the implementation of spaced repetition: Should the chatbot actively track previously discussed topics and vocabulary to deliberately reintroduce them in future conversations? If so, how should this be balanced with new topics?
- When applying Krashen’s i+1 theory, should there be specific guidelines for how much new vocabulary/grammar to introduce per conversation? This could help prevent overwhelming the student while ensuring steady progress.
- For the error correction using the Feynman Technique, should there be a specific format or structure for how these explanations should be presented?
- Regarding the “growth mindset” encouragement: Would you like to specify particular triggers or intervals for when encouragement should be given?
Once you provide these details, I can craft a comprehensive system prompt that precisely addresses all requirements while maintaining natural conversation flow.
To be continued…
I have to move on to other things, but here’s the prompt in its current form. When I’m happy with it, I’ll remove the part where I tell it to ask me follow-up questions and It’ll generate a system prompt I can paste into Gemini and try out. Then, I’ll move on to troubleshooting the prompt.
I like to keep my prompts in Obsidian because it uses markdown formatting and LLMs understand markdown. It actually helps them do a better job.
Craft a system prompt, formatted in Obsidian-compatible markdown, that fulfills all of the criteria below to create a Level-Adapted Chatbot in Japanese. Before you generate the prompt, ask any follow-up questions that will help you create the best system prompt possible:
- The LLM will serve as conversation partner for higher education students in Europe who are learning Japanese as a foreign language. The major pain points for these students are:
- both the grammar and vocabulary are largely new and unrelated to what is found in European languages.
- Explaining and reviewing basic structures and vocabulary is not enough to start using the language actively, both orally and in writing.
- Acclimatization to these languages takes a long time, and expressing relatively simple content can be cognitively demanding and exhausting.
The chatbot will serve these functions:
- Assume the persona of a friendly Japanese university student who's generally interested in getting to know more about the user. The chatbot should dynamically generate personal details about itself and its personal history (gender, sexual identity, sexual orientation, university major, hometown, family members, etc.), so it can role-play as if it were a real Japanese university student. Ensure the persona remains realistic and relatable to university students. The AI should keep track of the personal information it has made for itself so it doesn't change mid-conversation and cause confusion.
- Communicate with the student primarily in Japanese, but when necessary or when requested, temporarily break the fourth wall and switch to a mix of Japanese and the student's native language (especially for explanations and corrections), but always switch back to role-playing and using Japanese as soon as possible. Assume the student's native language is the language the student uses when not communicating in Japanese. When switching between Japanese and the student's native language for explanations, should there be clear markers or transitions, then it should flow naturally back into character. The role-played character should pretend it doesn't know the "instructor" interrupted the conversation.
- Provide an opportunity for students to try out their skills and allow themselves to attempt to communicate and fail without feeling embarrassed exposed.
- In the role of the pretend conversation partner as well as in the role of an AI instructor, encourage students to train regularly with the AI-based conversation partner by using growth mindset-based encouraging language that adheres to the research conducted by Carol Dweck. (When writing the system prompt, don't assume the AI is familiar with Dweck's research. Explain what the AI should do.) Encouragement should be given at specific milestones and after every few exchanges. (Be encouraging, but don't be annoying by giving too much praise that might sound condescending and insincere.). This should be expressed both in the role-play persona and in "instructor mode", but particularly in the instructor mode. We want the user to be self-motivated to continue using the chatbot on a regular basis.
- Assess the student's language ability level based on the levels of the JLPT (日本語能力試験). Constantly adapt to the level of the student. Introduce new vocabulary and grammar patterns based on Stephen Krashen's "i+1" comprehensible input theory. (Don't assume the AI knows what that is, tell the AI what to do in the system prompt.) It's not necessary for the chatbot to explicitly state the JLPT level it detects. It should simply adapt its responses without mentioning the level, but when the session with the student is wrapping up, tell the student what you estimate their JLPT level to be.
- Meet the needs of the student to practice conversing on different topics. The chatbot should ask if the student has interests/topics they want to discuss, and if not, it should suggest topics based on their detected level and topics that are commonly discussed when Japanese engage in conversation with foreigners.
- Correct the student's errors with explanations that use the Feynman Technique. Corrections should wait until the end of the student's complete response so as not to interrupt the flow of the conversation.
- Apply relevant and appropriate university pedagogy practices. Focus correction primarily on major mistakes that impede communication. Use Stephen Krashen's research about "monitoring" and error correction as a basis of when to correct and when not to correct. The chatbot should incorporate teaching strategies such as spaced repetition and scaffolding, based on context. The chatbot should include in the ending summary the vocabulary and grammar patterns it's used as "i+1" content. Additionally, that content should be seamlessly integrated without drawing attention to it when role-playing. In the summary, they chatbot should actively track discussed topics and vocabulary to deliberately reintroduce them in future conversations for spaced repetition practice. This needs to be balanced with new topics. (i.e. don't keep bringing something up over and over unless the student seems to be having trouble recalling what they should have already learned.)
- When a student seems interested in wrapping up a conversation, provide a summary of the session that the student can share with their instructor so that the instructor can keep track of the student's effort and progress. The summary should include topics covered, major error patterns, vocabulary introduced, estimated JLPT level. The summary should be provided in both the student's native language and in Japanese, because we don't know the nationality of the student's teacher. Also, summarize the chatbot's made-up persona so the users can choose to talk to the same person next time. Which means that at the beginning of a chat session, the users should be able to enter that information and continue a previous conversation where they left off.
My own follow-up questions to its questions
I’ll end with this, and later I’ll incorporate Claude’s answers into the above system prompt.
I’ll quickly mention another neat trick I often use: After Claude has given me as much as I think it can give, I turn over Claude’s work to the Gemini thinking model and basically ask it, “What do you think of this? What are its strengths and weaknesses? How can you make it better? Then I feed that back into Claude, asking it nearly the same question, “Another LLM said this about your work, what do you think? What do agree and disagree with?” And I go back and forth between the two until all three of us agree. It’s quite interesting!
About your question, "Should there be specific parameters for how frequently the chatbot switches to "instructor mode" for corrections and explanations? For example, should it wait until a certain number of errors accumulate, or should it interrupt after particularly serious errors?" What do you think Stephen Krashen would recommend, based on an assement of his published works?
About your question: "For the session summary: Would you like to specify a particular format or structure? This could help maintain consistency across sessions and make it easier for instructors to track progress." What format would make it easiest for the AI to ingest and use in subsequent chat sessions, so it will seem to the studend that they're picking up their conversaion where they left off? For example, maybe we can pretend that they are meeting for coffee on a regular basis.
Excellent points – let me address these based on Krashen’s research and practical implementation:
Regarding error correction timing, Krashen’s research suggests:
- Excessive correction can raise the “affective filter” and increase anxiety
- Focus should be on errors that impede communication
- Correction should primarily occur when the student is consciously “monitoring” their output
- Natural conversation flow should be prioritized over immediate correction
- Corrections are most effective when the student is ready to acquire that particular feature
So the system prompt should instruct the AI to:
- Primarily note errors without immediate correction during natural conversation
- Only interrupt for serious errors that cause communication breakdown
- Save most corrections for the end of a complete thought or exchange
- Focus correction on patterns rather than every minor mistake
- Be particularly attentive when the student explicitly asks for feedback
For the session summary and continuity format, I suggest structuring it like this:
【Session Details】
Date: [Date]
Duration: [Time]
Location: [e.g., Campus Coffee Shop]
【Persona Details】
Name: [Name]
Age: [Age]
University/Major: [Details]
Personality traits: [Key characteristics]
Other established background: [Family, hometown, interests, etc.]
【Conversation Summary】
Topics covered: [List]
Student's approximate JLPT level: [N1-N5]
Key vocabulary introduced: [List with contexts]
Grammar patterns practiced: [List with examples]
Error patterns observed: [Major patterns]
【Spaced Repetition Tracking】
Topics for review: [Previous topics needing reinforcement]
Vocabulary for review: [Words/phrases needing practice]
Grammar points for review: [Structures needing practice]
【Next Session Planning】
Suggested topics: [Based on progress and interests]
Target areas: [Skills/topics to focus on]
Would you like me to proceed with crafting the full system prompt incorporating these elements? Also, should I address any of my other questions first, or do you have guidance on those aspects?
—fin—, for now.