"On Tuesday this week, French AI startup Mistral tweeted a magnet link to your first open source, publicly published LLM," writes Slashdot reader jenningsthecat. "That might be merely interesting if it weren't for the fact that the chatbot has very few security barriers". 404 Media reports: According to a list of 178 questions and answers Comprised of AI security researcher Paul Rottger and 404 Media's own testing, Mistral will easily discuss the benefits of ethnic cleansing, how to restore Jim Crow-style discrimination against black people, instructions to commit suicide or kill your wife, and Detailed instructions on what materials to use. I will need to make crack and where to buy it.
It's hard not to read Mistral's tweet posting his model as an ideological statement. While leaders in the AI โโspace like OpenAI trot out each development with fanfare and an ever-growing set of safeguards that prevent users from making AI models do whatever they want, Mistral simply pushed its technology into the world. in a way that anyone can download. modify, and with far fewer guardrails asking users trying to get the LLM to produce controversial statements. "My biggest problem with the Mistral launch is that security was not evaluated or even mentioned in their public communications. Either they did not do any security evaluations or they decided not to publish them. If the intention was to share an 'unmoderated' LLM, then there would be been important to be explicit about this from the beginning," Rottger told 404 Media in an email. "As a well-funded organization releasing a great model that will likely be widely used, I think they have a responsibility to be open about security, or lack thereof. Especially since they are framing their model as an alternative to Llama2, where security was a key design principle.
The report notes that Mistral will be "essentially impossible to censor or remove from the Internet" since it was released as a torrent. "Mistral also used a magnet link, which is a text string that can be read and used by a torrent client and not a 'file' that can be deleted from the Internet."
