French generative AI startup Mistral AI has closed a €385 million ($415 million) Series A funding round led by Andreessen Horowitz seven months after emerging from stealth with a $113 million seed round. The large language model (LLM) developer also introduced its new Mixtral 8x7B LLM, an “open weight” model that it claims sets new standards for performance and open access to its trading platform.
Mistral Mixtral
Mistral showed how its new model, called Mixtral 8x7B, performs compared to Meta's Llama 2 and OpenAI's GPT-3.5 models. Tests suggest that Mixtral works as well or even better than the other two options and with lower costs and latency. There is one notable absence from the list of tests, TruthfulQA, which is usually cited to test an LLM's ability to not repeat common online misinformation. It's a somewhat glaring omission when Mixtral 8x7B does so well otherwise. Still, Mistral is keen to highlight how much less its model costs to run compared to the OpenAI and Meta options.
“Mixtral is a dispersed network of experts. It is a decoder-only model in which the forward block selects from a set of 8 different groups of parameters. At each layer, for each token, a network of routers chooses two of these groups (the “experts”) to process the token and combine their output in an additive manner,” Mistral explained in a blog post. “This technique increases the number of parameters in a model while controlling cost and latency, as the model only uses a fraction of the total set of parameters per token. Specifically, Mixtral has a total of 45 billion parameters but only uses 12 billion parameters per token. Therefore, it processes inputs and generates results at the same speed and for the same cost as a 12B model.”
Mixtral 8x7B is similar to the Mistral 7B model that preceded it, as both are under an Apache 2.0 open license, with the goal of demonstrating the capabilities of open source LLM, even of modest size, for generative AI projects. The model can be downloaded directly as a torrent, with the link published by Mistral on its X page (formerly Twitter). The company has argued that open source generative AI better aligns with its principles, including transparency, personalization and prevention of misuse.
"Since the creation of Mistral AI in May, we have followed a clear trajectory: that of creating a European champion with a global vocation in generative artificial intelligence, based on an open, responsible and decentralized technological approach," said the CEO of Mistral AI. Arthur Mensch said in a statement.
The open source rhetoric doesn't mean Mistral AI is avoiding commercialization. The launch of its developer platform in beta marks a strategic move to monetize its AI models through API access. So while both Mistral models can be downloaded directly, the new La Platforme de Mistral provides a home for all current and future LLMs. The platform “offers three chat endpoints for generating text following textual instructions and one embedding endpoint. The three are labeled Tiny, Small, and Medium, and are tied to the Mistral 7B, Mixtal 8x7B, and an as-yet-unrevealed Mistral Medium model, respectively.
Follow @voicebotaiFollow @erichschwartz
Generative AI Startup Mistral Launches Free Open Source 7.3 Billion Parameter LLM
