Broken Conversations – A Practical Guide for Improving Chatbot UX
Trust and Data Safety
Concerns user confidence in how their data is handled by both the brand and third-party platforms. Includes transparency around data usage, privacy, and training data for AI models.
Users have significant distrust when interacting with chatbots. They worry that their data is less safe than it would be if they filled out an online form or spoke to a human agent. There are also concerns that input data might be used to train the model, or that the LLM vendors behind AI-powered chatbots may not handle sensitive data as carefully as the chatbot provider promises.
Users need to be assured that their data is safe and that chatbot responses are reliable. Without that assurance, the result is a frustrated user.
Scenario 1: No visibility into data handling
A user interacts with a chatbot in a high-trust service context (such as banking) where the conversation requires sharing personal or confidential information. The user feels uncertain about the privacy and security of what they type into the chat. They do not know who can access it, whether it is retained, or whether it is used for AI training.
Scenario 2: Reservations about third-party involvement
A user interacts with a chatbot to resolve an issue related to their account, a transaction, or identity verification. During the conversation they are asked to provide personal or sensitive information. The user begins to wonder whether the chatbot is operated by an external provider, and who might have access to what they share.
Scenario 3: No corroboration that a response is valid
A user interacts with a chatbot to get information that directly affects a decision, such as pricing, eligibility, policy rules, delivery times, refund conditions, or technical specifications. The chatbot provides a confident answer, but the user is uncertain about its accuracy and wants to verify it before acting on it.
Scenario 1:
No visibility into data handling
A user interacts with a chatbot in a high-trust service context (such as banking) where the conversation requires sharing personal or confidential information such as account details, identity documents, or transaction references. The user feels uncertain about the privacy and security of what they type into the chat. They do not know who can access it, whether it is retained, or whether it is used for AI training.
Examples
Is the data secure?
The user is prompted to upload or enter their ID details. They have no way of knowing whether this information is stored securely or who has access to it.
Is the data private?
The user asks about a recent payment and is asked to provide personal details to proceed. The chatbot offers no reassurance about how that information is handled or kept private.
Will the data be used for AI training?
The user is concerned whether their input will be used to train AI systems or shared beyond the immediate interaction.
Why is this an issue?
The chatbot does not provide clear reassurance about data privacy and security:
- The user does not know who can access the data they provide.
- It is unclear to the user whether data is stored, and for how long.
- There is no transparency about whether data is used for AI training.
- Sensitive data requests are not contextualized or explained.
Why do we care?
This uncertainty creates hesitation and reduces trust in the chatbot channel. Clear, timely reassurance about how data is handled is essential to building trust and enabling task completion:
- Erosion of trust: The user may avoid using the chatbot for important tasks.
- Task failure: The user may abandon the flow or switch to phone support or another channel.
- Data quality: The user may withhold key details or deliberately enter vague or incorrect responses, reducing resolution speed and quality.
What is the remedy?
Users should feel safe when providing sensitive information:
- Provide proactive reassurance at the right moment (before sensitive input): Clearly state: who can access the data, whether it is stored, how it is protected and how long it is retained.
- Display an explicit AI training statement: For example: “Your messages are not used to train models” or clearly explain what is used and how.
- Offer visible privacy controls: Offer “Learn more”, retention period, and data deletion options where possible.
- Secure input methods: Masked fields, encrypted upload, or redirect to authenticated secure forms for documents.
- Use secure UI cues: Visual indicators (e.g. lock icons, secure session messaging) to reinforce trust.
- Minimize data collection: Ask only for what’s needed and offer safer identifiers (last 4 digits, ticket ID).
Are there any exceptions to this rule?
There are justified exceptions when providing these assurances about data handling might not be desired, such as:
- Over-reassurance in low-risk flows: Prominent privacy notices on simple, low-stakes interactions can feel disproportionate and may actually increase user anxiety rather than reduce it.
Scenario 2:
Reservations about third-party involvement
A user interacts with a chatbot to resolve an issue related to their account, a transaction, or the verification of their identity. During the conversation they are asked to provide personal or sensitive information. The user begins to wonder whether the chatbot is operated directly by their vendor or by an external provider, and who might have access to what they share.
Examples
Who is the chatbot operator?
The user is asked to confirm their account details. They hesitate, uncertain whether the chatbot is operated by the bank itself or by a third-party vendor.
Where will the data be processed?
The user is asked to verify their identity and questions whether this is handled internally or passed to external systems. The chatbot does not address the concern.
Whose branding is it?
The chatbot uses generic branding or language that gives no indication of who operates it, leaving the user unsure whether they are interacting with their bank directly or an outsourced solution.
Why is this an issue?
The chatbot lacks transparency about ownership and data access:
- It is unclear whether the chatbot is operated in-house or by a third party.
- The user does not know who can access or process their data.
- There is no explanation of how third-party involvement (if any) is managed.
- The system does not proactively address potential concerns about external access.
Why do we care?
The fear that information might be shared outside the main financial institution reduces user trust and may discourage users from using the chatbot for tasks requiring accuracy or privacy:
- Erosion of trust: The user may be less willing to share sensitive information if they suspect external access.
- Perceived risk: Third-party involvement may feel less secure to the user if not clearly explained.
- Task failure: The user may abandon the process rather than provide required data.
- Transparency: In high-trust domains like banking, the user may expect clarity about who handles their data.
- Data quality: The user may hold back details or provide incomplete information.
- Channel switching: The user may feel more comfortable switching to another channel and avoiding the chatbot entirely.
What is the remedy?
Clear, proactive transparency about ownership and data handling is essential to maintaining trust and enabling users to proceed confidently:
- Be transparent about who operates the chatbot: Clearly state whether it is operated by the vendor, a third-party or a combination and what that means in practice e.g. “Your information is securely handled by [Bank Name]. Our technology partners do not access your personal data.”
- Explain access and controls in plain language: State clearly who can view messages (bank staff only vs third-party), under what conditions, and how their access is logged.
- Address third-party involvement proactively: If external providers are used, explain their role and the safeguards in place.
- Give users a choice of channel: Offer escalation options (“Speak to a bank agent”, “Continue in secure messaging”) when sensitive data is required.
- Use consistent branding and tone: Reinforce that the user is interacting within the company’s trusted environment.
Are there any exceptions to this rule?
There are justified exceptions when providing these assurances about data handling might not be necessary, such as:
- Low-risk tasks: Information like branch hours, general FAQs etc. do not need heavy reassurance. Keep it lightweight to avoid unnecessary alarm.
Scenario 3:
No corroboration that a response is valid
A user interacts with a chatbot to get information that directly affects a decision, such as pricing, eligibility, policy rules, delivery times, refund conditions, or technical specifications. The chatbot provides a confident answer, but the user is uncertain about its accuracy and wants to verify it before acting on it.
Examples
How is this fee calculated?
The user asks about the cancellation fee. The chatbot provides a figure but offers no link to the terms and conditions or any other way to verify the answer.
What makes me eligible?
The user asks whether they are eligible for a plan. The chatbot confirms they are but provides no criteria, policy reference, or explanation. It then immediately asks if there is anything else it can help with, treating the matter as resolved.
Does this apply to me too?
The user asks about delivery times. The chatbot gives a timeframe but provides no indication of whether it is current, location-specific, or subject to any conditions.
Is this correct?
The user asks about refund terms. The chatbot summarizes the policy in its own words but provides no link to the official documentation, leaving the user with no way to verify what they have been told.
Why is this an issue?
The chatbot provides answers without verifiable sources or context but users want to see the chatbot responses backed up by data they can validate themselves on the website (or the internet):
- No links to supporting pages or official documentation.
- No indication of where the information comes from.
- No “last updated” or context (e.g. region, plan, conditions).
- Users cannot independently validate the response.
Why do we care?
Providing sources and context turns answers into trustworthy information. Lack of verifiable sources undermines trust:
- Decision-making: The user may need confidence before acting on important information.
- Transparency: Lack of sources may make the chatbot feel opaque or “black box”.
- Risk reduction: Especially in high-impact scenarios, the user may want proof, not just answers.
- Friction: The user may feel obliged to search for the validation of the response themselves.
What is the remedy?
Providing sources and context turns answers into trustworthy information:
- Cite sources with direct links to the relevant information e.g. “Based on our refund policy…”.
- Deep-link to the exact section (anchors / “jump to” highlights) instead of the generic homepage.
- Show supporting snippets (short quote or key bullet) so users can verify quickly.
- Add contextual qualifiers to indicate conditions such as location, plan, or timeframe to show that the user’s specific context has been taken into consideration.
- Add metadata like “Last updated on…” to help the user assess whether the information is current.
- Differentiate fact vs interpretation, for example: “Policy says X” vs “In your case, that means Y”.
Are there any exceptions to this rule?
There are justified exceptions when not providing proof might be acceptable, such as:
- Confidential/internal policies can’t be linked publicly.
- Dynamic content (prices, stock) may change quickly, making static references misleading.
- Low-risk or general knowledge queries For simple or non-critical questions, or when users are browsing rather than making decisions, users may not require sources.
In these cases, the chatbot should still provide a lightweight confidence aid (e.g. “Based on your account info” or “This can change. Confirm at checkout”).