Key Takeaways
- Two startups, Flower AI and Vana, created an AI model called Collective-1 using a new distributed training method.
- This method uses GPUs spread across the internet, unlike typical AI training concentrated in large data centers.
- The training included both public data and private user data from platforms like X, Reddit, and Telegram.
- This distributed approach could allow smaller companies, universities, or nations without massive data centers to build powerful AI.
- It potentially challenges the dominance of tech giants in AI development by using decentralized resources and new data sources.
Researchers are exploring a fresh way to build artificial intelligence models, potentially shaking up the current industry landscape.
Two startups, Flower AI and Vana, collaborated to train a new large language model named Collective-1. What makes this different is how it was built: using graphics processing units (GPUs) scattered worldwide, linked over the internet.
Flower AI developed methods allowing AI training to happen across many separate computers. This contrasts sharply with the usual approach where massive computing power is housed within single, expensive data centers.
Vana contributed data for the training, including private messages users volunteered from services like X, Reddit, and Telegram, alongside public information. This taps into data sources typically unavailable for AI model training.
While Collective-1 is relatively small compared to giants like ChatGPT, with 7 billion parameters, the team believes this distributed method can scale significantly.
Nic Lane from the University of Cambridge and cofounder of Flower AI mentioned they are already training larger models using this technique. He stated they plan to build models comparable in size to those from industry leaders later this year, potentially changing how AI is developed.
Currently, creating top-tier AI requires immense data and computing power concentrated in huge data centers, often packed with advanced GPUs. This favors the wealthiest companies and nations.
Distributed training could level the playing field. It might enable smaller organizations or universities to pool their resources and build sophisticated AI without needing their own giant data centers.
This approach involves rethinking how the complex calculations for AI training are divided and managed across geographically separate machines connected by standard internet lines.
Flower AI, along with academic partners in the UK and China, developed a tool called Photon to make this distributed training more efficient. They released Photon under an open-source license, allowing others to use the technology.
Vana’s role focuses on enabling users to share their personal data for AI training consciously. Co-founder Anna Kazlauskas explained this allows untapped private data to be used while giving users control and potentially ownership or benefits from the resulting AI.
Experts see potential benefits beyond just building models differently. According to Wired, specialists like Mirco Musolesi at University College London suggest this could unlock sensitive data for training, like in healthcare or finance, without the risks of centralizing it.
Helen Toner, an AI governance expert, finds Flower AI’s method interesting and possibly relevant for competition, though she notes it might initially lag behind the very cutting edge of AI development.
