{"id":763,"date":"2023-11-11T00:20:25","date_gmt":"2023-11-11T00:20:25","guid":{"rendered":"https:\/\/walterbot.ai\/?p=763"},"modified":"2023-12-02T22:06:00","modified_gmt":"2023-12-02T22:06:00","slug":"virtual-agents-on-messaging-apps-why-i-think-we-are-not-ready-for-this","status":"publish","type":"post","link":"https:\/\/walterbot.ai\/it\/2023\/11\/11\/virtual-agents-on-messaging-apps-why-i-think-we-are-not-ready-for-this\/","title":{"rendered":"Chatbots on messaging apps: are we ready?"},"content":{"rendered":"

**Per la versione Italiana del post, clicca QUI<\/a><\/span><\/em><\/p>\n

At the\u00a0<\/span>META Connect event 2023<\/span><\/a>, Mark Zuckerberg announced the introduction of “virtual assistants” on WhatsApp, Messenger, and Instagram.\u00a0<\/span>This news left me with more question marks than excitement because going beyond META itself as a company, I started questioning myself whether we, as “Humans” end-users are actually ready to have “generalists” chatbots in the palm of our hands. <\/span>Therefore, I decided to learn more about META’s new feature and the topic in general.<\/span><\/p>\n

Thanks to this exercise I learned about META’s best efforts to <\/span>build Generative AI Features responsibly<\/span><\/a> and\u00a0<\/span>their commitment to Privacy.<\/span><\/a>\u00a0However, few questions still need to be answered, and overall, my excitement didn’t really improve. I came to the conclusion that the current trend (not limited to META) of designing chatbots with personalities and background story, might work in the opposite direction of all the filtering systems that are being added to these tools to limit unwanted interactions. <\/span><\/p>\n

The difference between the tool and a human interaction should be constantly highlighted as I think it would help people developing a healthier level of trust in the tools, contributing to a more responsible and less harmful use of this technology. I see the value of AI in general and these tools for accelerating everyday routines. On the other side, I see no value but superficial, compared to the “risk” the end user might face in having a “digital assistant” pretending to be our best friend while gossiping our intimate information to third-parties.<\/span><\/p>\n

In this article, I will share what I have learned on the topic, highlight the open questions that I believe should be addressed before adopting such tools, discuss some concerns, and propose new directions to drive more responsible adoption.<\/span><\/p>\n

The Announcement:<\/h2>\n

META will soon introduce “virtual assistants” on WhatsApp, Messenger and Instagram. The “characters” will have different personalities, interests, opinions and background stories.<\/p>\n

Quoting the META website<\/a>:<\/p>\n

\"Fig.
Fig. 1 – Designed to be close. Source: https:\/\/about.fb.com\/news\/2023\/09\/introducing-ai-powered-assistants-characters-and-creative-tools\/<\/a><\/figcaption><\/figure>\n

 <\/p>\n

Apparently, the goal of these characters is to stimulate conversations with users for a more “immersive” interaction. Talking to them “should feel like talking to familiar people”.<\/p>\n

\"\"
Fig. 2 – “Familiar Interactions”. Source: https:\/\/about.fb.com\/news\/2023\/09\/introducing-ai-powered-assistants-characters-and-creative-tools\/<\/a><\/figcaption><\/figure>\n

 <\/p>\n

Ideally, users will include these virtual assistants in “group chat” or as direct contact to interact with. When added to groups, it can help organize events by retrieving information when asked to.<\/p>\n

\"\"
Fig. 3a – Source: https:\/\/about.fb.com\/news\/2023\/09\/building-generative-ai-features-responsibly\/<\/a><\/figcaption><\/figure>\n

 <\/p>\n

Information are expected to stay within the specific app, and not share with other users. Whether the learning from an interaction with a specific user will benefit other user’s chat is still unclear.<\/p>\n

\"\"
Fig. 3b – Source: https:\/\/about.fb.com\/news\/2023\/09\/privacy-matters-metas-generative-ai-features\/<\/a><\/figcaption><\/figure>\n

 <\/p>\n

Chat between users, family and friends will remain private and encrypted. Messages exchanged with the “Character” may be used for improving the service, meaning for re-training the AI model.<\/p>\n

\"\"
Fig. 4a – Source: https:\/\/about.fb.com\/news\/2023\/09\/introducing-ai-powered-assistants-characters-and-creative-tools\/<\/a><\/figcaption><\/figure>\n

 <\/p>\n

\"\"
Fig. 4b – Source: https:\/\/about.fb.com\/news\/2023\/09\/building-generative-ai-features-responsibly\/<\/a><\/figcaption><\/figure>\n

 <\/p>\n

The way the system works is well described in a picture taken directly from the META webpage:<\/p>\n

\"\"
Fig. 5 – Source: https:\/\/about.fb.com\/news\/2023\/09\/building-generative-ai-features-responsibly\/<\/a><\/figcaption><\/figure>\n

 <\/p>\n

As you can see, the “Human Feedback” block provides and essential part of data to train and fine-tune the model. This means that interactions are not only the final goal of the service but also the critical aspect for collecting data to keep evolving it.<\/p>\n

 <\/p>\n

Open questions:<\/h2>\n

Based on what I learned on the META website, below is the list of questions that I formulated when I first read about the announcement. Further down below, I report the questions that have been answered by the info retrieved on the META website:<\/span><\/p>\n

Q1) What are the benefits for the end-users to be incentivized to longer interactions with human-like chatbots?<\/span><\/p>\n

Q2) Will all the chats with the virtual assistant be used? If yes, for how long? Will the user be able to opt-out from having the chat used, or is it implicit that to avoid data collection, the only option will be not to leverage the service at all?<\/span><\/p>\n

Q3) What happens if all the META users start to ask the virtual assistant to seek for information instead of leveraging search engines: would that impact the free market? If yes, how and what alternatives will end-users and new start-up have?\u00a0<\/span><\/p>\n

Q4) How is META able to provide such services for FREE?\u00a0<\/span><\/p>\n

Q5) What are the risks (including unwanted answers or inappropriate conversations) associated with such services?<\/span><\/p>\n

Answered questions:<\/h2>\n

The questions for which I was able to find answers are the following:
\nQ7) When people interact over WhatsApp, Messenger, or Instagram, the conversations are encrypted. What would happen in this case where the interlocutor would be the same entity responsible for encrypting the exchanges?<\/span><\/p>\n

As stated on the META website (and reported in Figures 4a, 4b, and 5), only the user interactions with the chatbot will be used to re-train the model. Hence, we should expect any questions or conversations we will have with the chatbot to be shared to improve the model.<\/p>\n

Q8) What measures (if any) are in place to mitigate the dangerous unwanted effects?<\/span><\/p>\n

\n
\"\"
Fig. 6 – Source: https:\/\/about.fb.com\/news\/2023\/09\/building-generative-ai-features-responsibly\/<\/a><\/figcaption><\/figure><\/blockquote>\n

<\/h2>\n

Concerns and Observations:<\/h2>\n

At this point I envision 3 major areas of concerns related to Privacy, Safety and Business<\/strong>. It is important to note that the following considerations are not specifically related to the META product but to the overall idea of having “chatbots” to interact with at a general level, in the palm of user’s hands.<\/p>\n

Privacy concerns:<\/h3>\n

In March 2023, the\u00a0<\/span>Italian Privacy Guarantor blocked ChatGPT<\/span><\/a>\u00a0reporting – among the causes – the lack of transparency on how the system used data from user interactions. While it is clear that META will leverage users’s information to re-train the model and improve the service, determining the level of control (if any) the user will have over its data after being collected is still unclear. The only paragraph I was able to retrieve on this part on the META documentation is reported in Fig. 3b. While Fig. 3b shares tips on how a user will be able to delete a chat, there is no clear indication of how deleted messages will impact the model. Will it remember what it has learned from an interaction after a user has deleted it? What level of control will the user have on the entire data flow?<\/span><\/p>\n

When it comes to LLMs (Large Language Models)<\/span>, the current state of the art says that information used to train the model can’t be forgotten unless the model get re-trained from scratch without the selected information. Despite regulators have already discussed about “the right to be forgotten”, for which additional details and proposed solutions can be found HERE<\/a>, there is still a lot of skepticism. It would be like aiming at remove the salt from a sauce after it was cooked and blended together: the only way to be sure there is no salt, is to remake the sauce from scratch without the ingredient.<\/p>\n

These considerations become even more critical if we consider that a recent study<\/a> has shown how LLMs <\/span>are capable of\u00a0 capturing information about their interlocutor even when not directly expressed.<\/p>\n

<\/h3>\n

Safety concerns:<\/h3>\n

Recently a man from Belgium, committed suicide after chatting with an AI system. The machine seems to have influenced the user to end his life instead of suggesting to look for help. You can read more <\/span>HERE.<\/span><\/a> The man was a father of two and had a healthy job but was concerned about climate change. According to his Wife, her husband started interacting with the AI on the topic, but instead of finding hope, he seemed to have lost faith. A critical sentence reported in <\/span>a Euronews<\/span><\/a>\u00a0article, mentions:\u00a0<\/span><\/p>\n

\n

Consequently, he started seeing her as a sentient being and the lines between AI and human interactions became increasingly blurred until he couldn\u2019t tell the difference.<\/span><\/p>\n<\/blockquote>\n

While I appreciate META’s efforts to learn from these experiences and adopt a responsible approach (as stated\u00a0<\/span>HERE<\/span><\/a> and confirmed by Fig. 6), I have a few concerns about the complexity of the task that needs to be addressed. Machines can leverage human language to communicate but (still) lack the reasoning ability that would help to limit these cases. AI is able to generate text but does not know the difference between true and false or right from wrong. AI just know how to build meaningful sequences of words that are more or less related to the original queries. From this perspective, words can be used to express an infinite number of concepts. The larger the model, the more words and concepts it can generate. META virtual assistant seems to be extremely inclusive and leverages these large language models. However, there is no guarantee nor risk esteem that can help end users identify the level of maturity or reliability of these systems.<\/span><\/p>\n

The most critical aspect in what happened in Belgium is not related to the inability of the tool to filter out delicate questions, as it is on the human that has empathized with the tool. Empathizing is a human ability. It did not happen only in this case. How many people developed a kind of attachment with their first ever owned car? Or house? or cologne? or phone? Among humans, empathizing is a super-power because it facilitate the process of building relationships. Now this super-power might turn into a super-weakness if it leads us to forget the artificial nature of the AI system we might be interacting with. It is like if cars were designed with dashboards that would require 3 hands but a single driver to be properly maneuvered: humans would be penalized by their nature (only 2 hands) and risk of accidents would increase. The fact that these AI tools are being designed to close the perceptive gaps and blur the line between what’s synthetic and what’s human, might generate more risks than opportunity for healthy interactions in the long run. In this case it was an Adult, father of two, with a solid career and focused on a specific topic. Still, none of these aspect helped in avoiding the tragedy. What would happen if we put kids, with all their limitless curiosity and low experience in dealing with boundaries on the other side of the chat with an AI based tool purposely designed to “be friendly”?<\/p>\n

 <\/p>\n

Examples of unwanted or dangerous answers from LLMs are multiple and do not stop at the sad Belgian story. Below are some additional key references:<\/span><\/p>\n