Talks

One giant model, or many small ones? — Timnit Gebru on "frugal AI"

49 min read

Timnit Gebru · 2026 · 11:19

Summary

Gebru’s question: should AI be one giant model that does everything, or many small models, each built for one job and one community?

Her answer, in her own order:

  1. “AI” is not one thing. It is a changing bundle of methods. Some of them, like speech transcription, are useful and well defined.
  2. The “do everything” paradigm made speech recognition worse. Whisper, a general model, invents text that no one said, and doctors use it for patient notes. Her rule: start from the actual task.
  3. Big firms will not lead the other way. Their stolen or private data is their edge, so they have no reason to share efficient methods. Hope lies elsewhere.
  4. Where it already works. Te Hiku Media built Māori speech recognition and refused to hand its data to a US firm. DAIR is building its own small compute cluster.
  5. Small African-language groups are being squeezed. Investors told them to quit after Meta’s NLLB. Big labs told them to hand over data for peanuts.
  6. Her proposal: a federation. Small groups share data and compute, and present one interface to buyers.
  7. Close. The “machine god” project is a “secular religion” funded by billionaires. Task-specific tools are ordinary engineering. DeepSeek shows that constraint breeds cleverness, but also that “everybody’s imagination” has been “hijacked”.

Stance to note. She is not against AI. She is against one design for all uses, and against the way it is built (data, energy, labour). Her main language is moral and political (“stealing”, “exploiting”, “hegemony”). Part 3 separates the technical claims from the political ones, because they need different evidence.

Map — talking points

#Point
1Teaser: “machine God” builders steal data, harm the environment, exploit labour. Host: the “gigantic and hungry” claim
2”AI” is a shifting bundle. Chatbots now stand for all of it. Speech transcription is a well-defined, useful task, vital for oral cultures
3The “everything at once” paradigm. Old speech recognition made errors but not “hallucinations”
4Whisper: one giant model; used by doctors; invents text (“terror knife”)
5Lesson: one model for everything guzzles resources, adds new faults, and gives worse tools. Start with the problem
6Host: is universality a power play? Gebru: firms see data theft as an edge, so no incentive for efficiency
7Te Hiku Media: Māori speech recognition, refused licence to a US firm, “language back”, platform for other indigenous languages
8DAIR’s own cluster: about $400,000 once vs about $2 million a year in cloud (her figures). Pool unused GPUs
9Host: small networks are pluralism, and hegemony hates pluralism
10Lesan (Ethiopia) and Ghana NLP: Meta’s NLLB makes investors say “close up shop”. OpenAI offers peanuts for data
11Pressure to claim “all African languages” and global scale, against deep local context
12Proposal: a federation. Shared data, one application interface, curated data, low-resource models
13”Secular religion”: the machine god, pushed by billionaires; Mars and outer space
14Task-specific tools are ordinary engineering. The new thing is the opposite: the giant model
15DeepSeek: constraint breeds innovation, yet the same LLM paradigm. “Hijacked imagination”. Need content to unhijack it

Putting it together

The mental model in one paragraph. Gebru treats “one giant model” as a design choice with three kinds of cost. The first is technical: if the task is not defined you cannot test it, and a general text-generating decoder can invent words in a hospital note (Whisper). The second is social: the inputs (data, compute, labour) are taken from people who are not paid or asked, and the outputs concentrate power in a few firms that squeeze small, local groups. The third is ideological: the whole project runs on a belief in a machine god, which narrows the imagination about what AI could be. Her alternative is not a different algorithm. It is a different arrangement: small, task-specific models owned by the communities they serve (Te Hiku), their own compute (DAIR’s cluster), and a federation that gives them one front door and shared bargaining power. The best opposing view is that general models have repeatedly beaten specific ones on quality and cost, that big labs also push efficiency, and that small groups often build on the big models. The honest middle: the two approaches may complement one another, but the ownership, consent and pricing questions she raises remain open whichever way the technical contest goes.

Three tensions to keep open

  1. Quality or ownership? Is the case for small models that they work better, or that they belong to the right people? She uses both arguments. They need different evidence, and one can hold while the other fails.
  2. Open vs sovereign. Federation needs sharing, and data sovereignty needs control. The rules that join them (licences, governance) are the hard part.
  3. Efficiency vs total consumption. DeepSeek-style gains cut the cost per unit. They may raise total use (Jevons). Frugal design needs a way to keep the total in check, which the talk does not give.

Reading path

  • Start: The AI Now interview (PDF) to see the written version. Sutton, “The Bitter Lesson” (2019; two pages) to see the opposite view at its best.
  • Then: Bender, Gebru et al., “On the Dangers of Stochastic Parrots” (2021). Koenecke et al., “Careless Whisper” (FAccT 2024; I could not confirm this title via search, check it). Joshi et al. (2020) on linguistic diversity. IEEE Spectrum on Te Hiku (link).
  • Deeper: Schumacher, Small Is Beautiful (1973). Ostrom, Governing the Commons (1990). Kukutai and Taylor, Indigenous Data Sovereignty (2016) and the CARE principles. Gebru and Torres on TESCREAL (2024). Whisper paper (arXiv 2212.04356). Raji et al., “Everything in the Whole Wide World Benchmark” (2021). For the Indian angle: AI4Bharat and Bhashini’s own documentation†.

Part 1 — Before you read

Key terms

  • Machine god / “digital machine god”. Her name for the aim of building one system that can do any task, often called AGI (artificial general intelligence) or superintelligence. She treats it as a belief system, not an engineering goal.
  • General-purpose vs task-specific model. A general model is trained on huge mixed data and used for many jobs (chat, code, translation). A task-specific model is built and tested for one defined job, such as Māori speech to text. Watch out: “small” and “task-specific” are different things. A small model can be general, and a large model can be fine-tuned for one task.
  • Frugal AI. The host’s term; AI Now’s title. It means AI that uses little data, energy and money. Gebru says the idea is not new. What is new is the opposite, the giant model.
  • Speech recognition (ASR) vs “hallucination”. ASR turns speech into text. In an LLM, a hallucination is fluent output that is not grounded in the input. She says that classic ASR made mistakes but not this kind of invention. She also calls “hallucination” “not a good term”.
  • Low-resource language. A language with little written or recorded data online. The term is about data, not about how many people speak it. Hindi is large in speakers and still lower-resource than English.
  • Data sovereignty / “language back”. The claim that a community should control data about itself. Keoni Mahelona’s “language back” echoes the “land back” campaign: technology based on a language should first serve its speakers.
  • Cluster. A group of linked computers with GPUs (graphics chips used for AI). She means a small, self-owned data centre.
  • Federation. Independent groups that keep their own control but share data, tools or an interface. Compare a union of co-operatives.
  • Pluralism vs hegemony. Many different systems side by side, against one system with power over the rest. The host’s frame.
  • Export controls. US limits on selling the best AI chips to China. DeepSeek is her example of what constraint produces.

People and groups named (a reading list)

NameWhy they come upStart with
OpenAI (Whisper)Her failure case: a giant general model used in medicineThe Whisper paper, arXiv 2212.04356 (Dec 2022)
Te Hiku Media (NZ), Keoni MahelonaThe working model of community-owned language AINVIDIA write-up; IEEE Spectrum
DAIRHer institute. Own cluster; “many models for many people”DAIR site (the project page I tried returned a 404; a search snippet describes it, see debate below)
Lesan (Ethiopia), Ghana NLPSmall African-language groups under pressureTheir own sites†
Meta, “No Language Left Behind”The model that made investors doubt small firmsNature paper, June 2024 (press coverage)
DeepSeekEfficiency under export controlsSee debate below
The host’s frame: “hegemony hates pluralism”Political-economy reading of the whole issueidea 3

Gebru’s own frame

Gebru was a co-author of On the Dangers of Stochastic Parrots (2021), the paper that asked whether language models can get “too big” and was at the centre of her exit from Google in December 2020 (the exit date is confirmed in the AI Now introduction; the paper’s details are from memory)†. Earlier she co-wrote Gender Shades (2018, with Joy Buolamwini) and Datasheets for Datasets (2018). Her 2022 institute, DAIR, funds “community-rooted” research. This talk applies her older argument (big models are risky and badly specified) to a constructive proposal (a network of small, local ones). A search snippet describes DAIR’s Many Models for Many People project as aiming to counter “one giant model for everything” by backing small community organisations (DAIR, snippet only; page not opened).

The current debate (2025–26)

  • The interview itself. AI Now published it in Feb 2026 as part of Reframing Impact, with a “frame” (the machine-god paradigm creates new problems and starves imagination) and a “reframe” (stitch together frugal efforts, federate them) (AI Now PDF). It was made for the India AI Impact Summit run-up. That is relevant to India, where many languages and state programmes face the same choice.
  • Whisper in medicine. An AP investigation (Oct 2024) found Whisper invents text. Researchers led by Allison Koenecke (Cornell) with Mona Sloane (Virginia) studied thousands of short clips from TalkBank. One report says 187 hallucinations in over 13,000 clear audio snippets, and that nearly 40% of the hallucinations were harmful or concerning. OpenAI advises against use in “decision-making contexts” (Scripps/AP, WEAR-TV). These are news summaries, not the paper itself. The 40% and 187/13,000 figures come from a secondary report.
  • Te Hiku’s results. An early Te Hiku model, built on Mozilla’s DeepSpeech, reached a 14% word error rate. A later NVIDIA NeMo model reports about 92% accuracy for te reo and about 82% for mixed English–Māori speech. The two numbers are from different model generations and different tests, so do not compare them. Te Hiku released data under a Kaitiakitanga licence limiting use to Māori benefit (IEEE Spectrum, NVIDIA, via search summaries).
  • Small African-language models. Lelapa AI (South Africa, founded 2022) released InkubaLM, a 0.4-billion-parameter model for five languages (Swahili, Yoruba, isiXhosa, Hausa, isiZulu). It claims results close to larger models on translation and question answering (Lelapa, arXiv 2408.17024). The claim is the lab’s own.
  • DeepSeek and the cost of training. DeepSeek’s V3 paper (Dec 2024) said about $5.6 million on 2,000 H800 chips. Critics said that excluded earlier work and research costs. A Nature paper in Sept 2025 gave R1’s own training cost as $294,000 on 512 H800s, while admitting A100 chips were used in earlier stages. US officials and Nvidia disagreed on which chips DeepSeek had (Al Jazeera, BNN Bloomberg). Note: these costs are for final runs of very large models. DeepSeek is still a large-scale lab.
  • The scaling dispute. The “Bitter Lesson” view (general methods that use more compute win) is still the strongest case against her. Critics say the essay is too binary and describes the past more than it predicts the future (summary). The scaling-law papers (Kaplan 2020 vs Chinchilla 2022) disagree on how to split a compute budget between model size and data (Pearce & Song).
  • A related Gebru interview (Wired, Sept 2026). A secondary report says she told Wired’s Lauren Goode that “machine god” extinction talk distracts from real harms such as autonomous weapons, data-centre climate damage and layoffs (AI Weekly summary). I did not read the Wired piece, so treat the content and date as (not confirmed). It is a different interview from this one.

The ideas

Gebru spoke for 11 minutes, so the talk is a short summary of long arguments. Seven families of ideas meet in it. The depth here is to show you what sits under each short claim.

 SCALING HYPOTHESIS ◄──── her main opponent ────────────┐
 Sutton 2019 · Kaplan 2020 · Chinchilla 2022            │
                                                        ▼
 CRITIQUE OF SIZE ─────►  ┌──────────────────────────────────────┐ ◄─── APPROPRIATE
 Stochastic Parrots 2021  │  GEBRU: one giant model is a bad     │      TECHNOLOGY
 Strubell 2019 · Raji     │  design and a bad politics; build    │      Schumacher 1973
 "everything benchmark"   │  small, specific, community-owned    │      Illich · Winner
                          └──────────────────────────────────────┘
                            ▲            ▲              ▲
   INDIGENOUS DATA          │  LOW-RESOURCE NLP         │  POLITICAL ECONOMY
   SOVEREIGNTY              │  Joshi 2020 · Masakhane   │  Compute concentration
   Te Mana Raraunga · CARE  │  NLLB · Lelapa · Bhashini │  Data colonialism · labour
                                                        │
              IDEOLOGY OF "AGI"                         │
              Bostrom 2014 · TESCREAL (Gebru & Torres 2024)

Where each school of thought is covered:

  • The scaling hypothesis — the case she argues against: idea 1
  • The critique of size — Gebru’s own body of work: idea 2
  • Specification and engineering — “start with the actual thing”: idea 2
  • Appropriate technology and “small is beautiful”: idea 8
  • Indigenous data sovereignty — Te Hiku and “language back”: idea 4
  • Political economy — compute, data, labour and capture: idea 3
  • Low-resource languages and language technology: idea 6
  • The ideology of “AGI” — the “secular religion”: idea 8

1. “AI” is a bundle, and speech recognition shows what is lost

In short. Gebru says “AI” is not one set of methods. It is a bundle of things, and some are useful. Speech recognition is her example: a well-defined task, speech in and text out, which matters for oral cultures. The new approach says use as much data and as big a model as possible, with no single task. That swaps a clear field for one giant model.

🎙 In the talk. Gebru: “AI is not like a coherent set of methods or technologies. There’s a whole bunch of things that go in and out of being called AI”. Today it “is synonymous with these chat bots and image generators”. Inside the bundle, “some of them I think are useful”, and the example is transcription: “a very well-defined task where the input is speech and the output is text”. It matters because “we have a lot of cultures in the world that are oral cultures”. In the past, speech recognition was its own field with its own work. In the new age, “use as much data as possible, use as big models as possible and don’t just try to have one task”. Old systems made mistakes, but “this idea of so-called hallucinations… was not a problem with speech recognition”.

📚 Deeper theory

  • “AI” has always been a moving label. John McCarthy coined it in 1955–56. Tasks that become routine (OCR, chess, spell-check) stop being called AI, a pattern known as the AI effect. Her point is that this vague label lets a chatbot’s reputation spread to unrelated tools (idea 2).
  • Speech recognition is a good test case because it has a clear ground truth (what the person said). Word error rate can be measured. So you can show whether the general model beats the specific one (Roots, idea 2).
  • Oral cultures. Walter Ong’s Orality and Literacy (1982) is the standard text on cultures that keep knowledge in speech. For such communities, speech-to-text is a way into written services. This is the equity argument for ASR.
  • Her “no hallucinations before” claim is strong. Classic ASR did produce wrong words, and some systems produced odd output on silence or noise. What changed is that a generative decoder can produce fluent, plausible text, so errors are harder to spot (idea 2, point 3). That is a reasonable reading but a claim about degree.

📚 Roots: The scaling hypothesis — the case she argues against

The problem. For decades AI researchers asked whether to build in human knowledge (rules, features) or let the machine learn from data. The scaling view says: use more data and compute, and results keep improving in a predictable way.

The point. She does not deny that scale produced capability. She argues that (a) the goal (one model for everything) is badly specified, (b) its costs fall on others, and (c) no one in the scaling camp has an incentive to explore the alternative.

The chain of thinkers

  1. Rich Sutton, “The Bitter Lesson” (2019). A short essay. Looking back at 70 years, he argues that general methods that use computation beat methods that use human insight, in chess, Go, speech recognition and vision. The “bitter” part is that researchers keep hoping their cleverness will win. Key point for this talk: speech recognition is one of his own examples. Hand-built phonetic knowledge lost to statistical methods that used more data and compute. Gebru’s speech example is on his turf.
  2. Kaplan et al. (OpenAI, Jan 2020), “Scaling Laws for Neural Language Models”. Test loss falls as a smooth power law as model size, data and compute grow. Their advice: put most of a bigger budget into model size.
  3. Hoffmann et al. (DeepMind, March 2022), “Chinchilla”. Corrected the advice: for a given compute budget, size and data should grow together. The two papers produce different exponents (about 0.73 vs 0.50 for parameters, per Pearce & Song). Key point: later work found that training a smaller model on much more data gives a model that is cheaper to run. So even inside the scaling camp, “small but well trained” became an engineering goal.
  4. Radford et al. (OpenAI, Dec 2022), Whisper. The model she criticises is a direct product of this view. It was trained on about 680,000 hours of audio in many languages and tasks, using “weak supervision” (noisy transcripts from the web), and aimed to work “out of the box” without per-dataset tuning (from the paper’s title and summary, search result). Her complaint is what that design costs.
  5. Wei et al. (2022) on “emergent abilities”, and Schaeffer et al. (2023) “Are emergent abilities a mirage?” Large models seem to gain new skills at certain sizes. The second paper argues many of these jumps come from how success is measured. (†) This matters because “emergence” is the hope behind “machine god” talk.

Where others push back.

  • Whisper and its successors improved speech recognition for many languages and accents, and made decent ASR free for small developers. For many users a general model works well enough, and cheaply.
  • The hallucination rate is low in the Cornell study (187 in over 13,000 snippets, from the secondary report) and can be reduced with filters, silence detection and human review. The right fix may be controls, not a new architecture.
  • Sutton’s view: the history of ASR is the history of general methods beating hand-built ones (Roots).

What she leaves out.

  • Numbers. She gives no word-error rates for Whisper vs a task-specific system on the same data. Her case would be stronger with one head-to-head test.
  • That Whisper is also open and can be fine-tuned. Many small groups use it as a base. The line between “giant” and “small” is less clean than her contrast suggests.

Check yourself

  1. Why is speech transcription a good test of “task-specific vs general”?
  2. In the Roots of idea 2, point 3, why can a general decoder write words nobody said?
  3. Name one strong reply from the scaling side.

2. The Whisper case — one model, new failures

In short. Whisper is one giant model that does many things, and doctors now use it to transcribe patient notes. It makes things up, such as turning a harmless sentence into a killing. Her lesson is that if you do not start with the actual problem, you do a bad job at everything. She calls a universal tool a fool’s errand.

🎙 In the talk. Fast forward to 2021. OpenAI has something called Whisper… like one giant model that does a bunch of different things. It’s been integrated into ChatGPT, and now we hear that doctors are using Whisper to transcribe patient notes. Whisper has been found to make up a whole bunch of things. She retells an example that turns a harmless sentence into “he was holding a terror knife and he killed a bunch of people”. Her lesson: “One giant model for everything not only just guzzles everything in the environment, but also has introduced problems we didn’t even have before. And also results in subpar tools for many people around the world”. And: “If you don’t start with the actual thing you’re trying to solve for, you end up doing a bad job at everything”. Then, on the general tool itself: “isn’t it wonderful we have a universal tool that’s kind of a fool’s errand”.

📚 Deeper theory

  • What was checked. The AP report and the Cornell study (Koenecke, Sloane) back the core: Whisper invents text, and hospitals use tools built on it. One secondary report counts 187 hallucinations in over 13,000 clear snippets, with nearly 40% judged harmful or concerning. OpenAI itself says not to use it in high-stakes decisions (Scripps/AP). The detail that the errors fell more often on certain speakers (people with aphasia, in the study’s data) is†
  • “Integrated into ChatGPT.” Whisper powers voice input in the apps and is sold by API. That the same model sits behind a chat voice mode and also behind third-party hospital tools shows her theme of one component used for everything†.
  • Harm model. A hospital note is a record. A fluent false line, such as an invented drug name (the search report lists “hyperactivated antibiotics”), is the worst kind of ASR error, because a reader sees nothing wrong. A wrong word is easy to catch. An invented sentence looks like a real one.
  • Responsibility. The tool maker says “don’t use it for this”; the hospital vendor does it anyway. Gebru’s wider writing puts the duty on builders and deployers. The “intended use” idea from safety engineering (Roots, point 4) is the missing control.
  • “Guzzles everything in the environment.” A general model costs energy to train and to run. But per transcription, Whisper’s larger versions are not the biggest models in use, and smaller Whisper versions exist. Her environmental claim here is general, and the Whisper case does not by itself prove it.

📚 Roots: The critique of size — Gebru’s own body of work

The problem. If bigger is better, who pays, and what is lost?

The point. Her first line, “stealing data, killing the environment, exploiting labor”, is a short list of the costs in the Roots of ideas 2 and 3.

  1. Strubell, Ganesh and McCallum (2019), “Energy and Policy Considerations for Deep Learning in NLP”. One of the first papers to estimate the carbon cost of training large models. (†)
  2. Bender, Gebru, McMillan-Major and Shmitchell (FAccT 2021), “On the Dangers of Stochastic Parrots”. Argued that ever larger models bring four kinds of cost: environmental, financial (so only rich labs take part), data (huge web scrapes hide bias and cannot be audited) and a research cost (effort goes to scale, not to understanding). The “parrot” image says a language model stitches together forms without access to meaning. (Summary from memory; the authorship and venue are well known but I did not re-check them for this page.)
  3. Raji, Bender, Paullada, Denton and Hanna (2021), “AI and the Everything in the Whole Wide World Benchmark”. Argues that “general” benchmarks pretend to test general ability while hiding what is being measured. Key point: this is the formal version of her line “if you don’t start with the actual thing you’re trying to solve for, you end up doing a bad job at everything”. A task you cannot define, you cannot test.
  4. Mitchell et al. (2019), “Model Cards”, and Gebru et al. (2018/2021), “Datasheets for Datasets”. Documentation practices that say who a model or dataset is for. They only make sense when a tool has a defined use. (From memory.)
  5. Luccioni and colleagues (2023–24) on the footprint of inference, the day-to-day running cost, not only training. (†)

📚 Roots: Specification and engineering — “start with the actual thing”

The problem. What does it mean for a tool to be good? Engineers answer: good for a stated purpose under stated conditions.

The point. Her central rule: the task comes first. Her evidence is the Whisper failure. The wider theory says why a vague task makes evaluation weak.

  1. A bridge is designed for a load. Gebru’s own example elsewhere is a bridge built by people who are accountable for it (from a secondary report, AI Weekly, not this talk). In this talk she says only: “a task-specific tool is just a concept in anything we build in engineering”.
  2. The Unix idea (1970s): one tool, one job, joined by simple interfaces. Compare her proposal of small groups behind “one application interface”. (Analogy mine, not hers.)
  3. How speech recognition used to work. Classic systems used separate parts: an acoustic model, a pronunciation dictionary and a language model, tied by a search. Whisper is one network with a text-generating decoder. That decoder is a language model, and a language model is built to produce likely text. This is the technical reason a model can write fluent words nobody spoke, while a classic system tends to output the wrong word or nothing. (General background; not from the talk, and the full explanation of Whisper’s failures is still being researched.)
  4. Safety engineering and “intended use”. In medicine, a device is cleared for a stated use. A general model used for patient notes has no such clearance. OpenAI’s own warning against “decision-making contexts” shows the gap.

Where others push back.

  • Case for scale: The Whisper paper argued that training on diverse, noisy data gives robustness, since it does not break when the accent or noise differs from a lab dataset. Task-specific systems trained on narrow data often fail outside their test set. Generality may reduce some real-world failures.
  • Case for fixes: Hallucination is a known problem for all generative models, and many teams work on it. A single point of failure can be patched once for all users, which is hard to do across hundreds of small systems.
  • Case for caution about her example: The harm came partly from deployment choices, not just architecture. A small neural ASR with a generative decoder could fail the same way.

What she leaves out. Evidence that a task-specific medical ASR system performs better and fails more safely. Such systems exist (dictation tools for doctors have long existed), and a comparison would settle the point†.

Check yourself

  1. What is the exact claim she makes about hallucination in older ASR, and what is the technical reason for it?
  2. Which parts of the Whisper story are verified, and which part of her retelling did I not find?
  3. What would a fair head-to-head test look like?

3. Why the giants will not go small — incentives and data

In short. Asked whether universal tools exist to concentrate power, Gebru says the big firms have no reason to look for cheaper methods. They see their data, including data they should not have, as an edge. So she does not expect research on doing things efficiently, and she looks elsewhere for ideas.

🎙 In the talk. The host asks whether the push for universal tools exists “because it allows for a concentration of power”. Gebru: “the industry has absolutely no incentive to look at less resource intensive things for anything because they view their stealing of data and having and using private data as a competitive advantage. ‘Cause they have all the data”. So “there’s not gonna be any research in these kinds of places of how to do something super efficiently and then produce and give that research away… I’m not looking to those big players for any kind of inspiration here”.

📚 Deeper theory

  • Incentive logic. If data is a moat, efficiency research that lets others do the same with less data weakens the moat. This is standard market-structure reasoning (Roots).
  • “Stealing.” This is a legal and moral claim. Many court cases over training on copyrighted work are active in 2025–26. I did not check their status for this page. Her word is an accusation, and its legal truth is contested and case-by-case.
  • But big labs do research efficiency. Distillation (Hinton, Vinyals and Dean, 2015), quantisation and mixture-of-experts are standard in large labs, because inference costs money. DeepSeek is the example she herself raises. So “no research on efficiency” is too strong. Her narrower point, that they will not research small, task-specific, community-owned systems and give them away, is more defensible.
  • Academic capture (Roots). If research needs corporate compute and funding, independent work on frugal systems is starved. Hence her call for independent clusters.

📚 Roots: Political economy — compute, data, labour and capture

The problem. Who controls the inputs of AI (data, chips, cloud), and who does the hidden work?

The point. The squeeze on Lesan and Ghana NLP is a live case of capture: investors, customers and labs all push small firms toward the big one’s terms.

  1. Couldry and Mejias, The Costs of Connection (2019), “data colonialism”. Taking human life as raw material for capital echoes older colonial extraction. (From memory.)
  2. Birhane, “Algorithmic Colonization of Africa” (2020); Mohamed, Png and Isaac, “Decolonial AI” (2020). African and global-south scholars on how AI products built elsewhere reshape local life. (From memory.)
  3. Meredith Whittaker, “The Steep Cost of Capture” (2021). University AI research depends on corporate funding and compute, which shapes what gets studied. This is the logic behind her claim that big firms will not research efficiency for others. (From memory.)
  4. Gray and Suri, Ghost Work (2019); reports on data-labelling workers in Kenya and elsewhere (2023). The “labour” in her first line: low-paid people label data and filter harmful content. DAIR runs a Data Workers’ Inquiry project†.
  5. Karen Hao, Empire of AI (2025). A journalist’s history of OpenAI as an empire-like project. (†)
  6. Compute as the chokepoint. AI Now’s 2023 work on “computational power” argues that chips and cloud are the real concentration points†. Gebru’s cluster is a small answer. Her claim: about $400,000 once against almost $2 million a year in cloud. This is a rent vs own calculation. It holds only if the machines stay busy, which is why she adds pooling with Te Hiku and others “who don’t even use their GPUs that much”.
  7. Commons and federation: Elinor Ostrom, Governing the Commons (1990). Communities can run shared resources without a state or a market, under rules they set. This is the best theory for her pooled-compute idea. (Link is mine; she does not name Ostrom.)

Where others push back.

  • Large labs have released open models (Meta’s Llama, Mistral, DeepSeek, Google’s Gemma, OpenAI’s open-weight models) that small groups use as a base. Efficiency gains from the labs do reach others.
  • Efficiency is commercially valuable, so firms have a strong profit incentive to cut cost per query. A “moat” argument ignores this.
  • The competitive advantage may be compute and talent more than raw data.

What she leaves out. A distinction among firms. Some large labs behave differently, and some small firms also scrape data. Her “stealing” claim is also not backed in the talk by examples.

Check yourself

  1. Why does she say big labs have no incentive to research efficiency?
  2. Name one way large labs do research efficiency anyway.
  3. What would you need to know to test the “stealing” claim?

4. Te Hiku Media — small, owned, and unwilling to sell

In short. Te Hiku Media built speech recognition for Māori, and refused to license the data to an American firm. Their rule is that the work must serve the Māori people first. Gebru finds this inspiring because they want to serve their own language and help others do the same, not build one machine God. She says others can adapt parts of it to their own context.

🎙 In the talk. “They built the speech recognition system for Māori language. They gathered this data. An American company wanted to license the data, and they said, absolutely not because everything we do has to serve the Māori people first”. Gebru reports their reason: “You guys kind of beat this language out of our grandparents, and now you wanna sell it to us as a service”. Keoni, she says, talks about “a language back campaign that is similar to the land back campaign… a part of the technology that is based on a specific language should first and foremost serve the speakers of that language”. They also built a platform “to help speakers of smaller indigenous languages create tools that serve them”. For her it is inspiring because “they’re not trying to build the one machine God… They’re only concerned about their own language and they’re supporting other people”. Further: “you can look at aspects of what they’re doing and adopt it to your context”.

📚 Deeper theory

  • What search checked. Te Hiku built a Māori ASR on Mozilla’s DeepSpeech, with over 300 hours of labelled speech gathered in about ten days and an early word error rate of 14%. A later NeMo-based model reports about 92% accuracy for te reo, and 82% for mixed English–Māori (via NVIDIA and IEEE Spectrum summaries; not checked at source). Mahelona is the chief technology officer.
  • The licence is the legal tool for her principle (Roots). It means the data is not “open” in the Silicon Valley sense, and that is the point. CARE says Collective benefit and Authority to control come before open access.
  • The history behind the refusal. Te reo Māori was suppressed in schools in New Zealand (the “grandparents” line). Technology that sells it back as a paid service repeats the harm. The revival of te reo since the 1980s (kōhanga reo language nests) is the social base of the project†.
  • Why it works as a model. The group has trust (community members donate recordings), a clear purpose (te reo), and in-house skill. These are not easy to copy. Gebru says “adopt aspects”, which is more modest than “copy”.
  • Scale limit. Te Hiku’s clear gain comes from one language with a motivated community and a team. A language with few speakers or no organised body cannot do the same without outside help, which is why she turns to federation.

📚 Roots: Indigenous data sovereignty — Te Hiku and “language back”

The problem. Who owns the data about a people, and who profits from it? Research and tech have often taken data from communities and given nothing back.

The point. Gebru reports Mahelona’s reply to the US firm: “everything we do has to serve the Māori people first”. The licensing refusal is the practical proof that small can mean owned, not only cheap.

  1. Linda Tuhiwai Smith, Decolonizing Methodologies (1999). A Māori scholar shows how research has served colonisers. Foundation for later work. (From memory.)
  2. First Nations OCAP principles (Canada, late 1990s) and Te Mana Raraunga (Māori Data Sovereignty Network, 2015). Communities assert ownership and control. (From memory.)
  3. Kukutai and Taylor (eds.), Indigenous Data Sovereignty (2016). Collects the field. (From memory.)
  4. CARE Principles (Global Indigenous Data Alliance, 2019): Collective benefit, Authority to control, Responsibility, Ethics. They were written to sit beside the FAIR data principles. (From memory.)
  5. Te Hiku Media’s Kaitiakitanga licence. Kaitiaki means guardian in te reo Māori. The licence limits use of the data to benefit Māori (IEEE Spectrum, via search summary). A student blog also claims profit-sharing terms; that source is weak, so this detail is unconfirmed. A conference abstract says the team collected over 300 hours of labelled speech in about ten days, with community help (via search; not checked at source).

Where others push back.

  • Open-data critics: Restrictive licences may slow research and stop other communities from benefiting. The counter is that community control is the aim, not a side effect.
  • Resource critics: Even Te Hiku used NVIDIA hardware and open-source toolkits made by big firms. A fully independent stack does not exist. Gebru does not claim it does.
  • Quality: A model trained on little data can be fragile in the wild. The reported 92% is for a given test; results on other speakers or noisy audio are unknown†.

What she leaves out. How Te Hiku pays for its work (grants, government, commercial work?), and what the cost to the community is. Sustainability is the hard part of the model.

Check yourself

  1. What does “language back” mean, and what does it copy?
  2. What is a Kaitiakitanga licence, and why is it not “open data”?
  3. What stops Te Hiku’s model from being copied everywhere?

5. DAIR’s cluster and pooled compute

In short. DAIR is building its own small cluster rather than renting cloud compute. She says a one-time cost of about $400,000 would replace almost $2 million a year in cloud fees. The wider idea is to pool resources, including idle GPUs, so peers can support each other. She sees a different ecosystem forming around this.

🎙 In the talk. “At DAIR we’re building our own cluster, think of it like a small data center because we don’t wanna use cloud computing resources, and we want to support our peers who don’t wanna use AWS or Google Cloud”. “A one-time investment of, let’s say, $400,000, which is a lot, would give us an equivalent cluster of what would’ve cost us almost $2 million per year to use in cloud computing services”. The question: “How can we share resources across all of us… and pool our resources to support each other”, including “some other people who don’t even use their GPUs that much”. Then: “There is an alternate sort of ecosystem that’s building around this idea that no, we don’t have to do the same things that these people are pushing”.

📚 Deeper theory

  • Rent vs own. The numbers (her estimate, unverified) imply payback in a few months if the cluster is as busy as the rented one. In practice, owned hardware has extra costs: power, cooling, space, maintenance staff, and ageing chips. And rented clouds sell flexibility, such as scaling up for a burst. The calculation is plausible for steady use and weaker for spiky use.
  • Pooling. If several groups each use their GPUs 20% of the time, a shared pool can serve more work from fewer chips. This is classic statistical multiplexing, the same logic as a power grid or a co-operative bank. Ostrom’s work (idea 3, point 7) says such pools need clear rules on who can use what, how to settle disputes, and how to handle free-riders.
  • Sovereign compute. Governments now fund “sovereign AI” capacity (India’s IndiaAI Mission is one)†. Her version is smaller and civil-society run.
  • Fit for task. A small model for one language needs far less compute than a frontier model. That is what makes a $400,000 cluster useful. It cannot train a frontier model, and she is not trying to.

Where others push back.

  • Cloud economics: Large providers get better chip prices and keep utilisation high. A small owner may not beat them once staff time counts.
  • Frontier gap: If small groups can only train small models, they stay below the capability line that buyers want. This feeds the pressure to claim wider coverage that she describes. (The AI Now text, not this video, adds that customers told these groups to “come back when you have more linguistic coverage”.)
  • Chip supply: Buying and running GPUs is hard for small organisations in many countries.

What she leaves out. The running costs, the cluster’s size and specification, and who uses it. Without them the $400,000 vs $2 million line is a headline, not a model.

Check yourself

  1. Under what condition does owning beat renting?
  2. What is the rule you need for a shared pool to survive?
  3. What can a $400,000 cluster not do?

6. The squeeze on small language groups — Lesan, Ghana NLP and the “closing up shop” problem

In short. Gebru works with Lesan, for Ethiopian languages, and Ghana NLP, who had similar complaints. After Meta announced a model for 200 languages, investors told them to close up shop. Clients and investors also push them to claim all languages and global scale. She says their value is local context, which a model built for scale does not have.

🎙 In the talk. Gebru works with “one called Lesan, that focuses on Ethiopian languages, and another one called Ghana NLP” and found “they both had kind of similar complaints”. Meta “came out with an announcement of a model called No Language Left Behind… one model that performs automatic translation across 200 languages. Including 55 African languages for which there was, before, no state-of-the-art translation system”. Then “a number of potential investors in these smaller organizations… literally told them to close up shop. They were like Facebook has solved it”. From OpenAI and others: “we’re gonna make our models better in your language. You’re better off collaborating with us and supplying us data for which we’re gonna pay you like peanuts”. For Lesan, “your contextual knowledge is not about South Africa, it’s about Ethiopia and that political situation and how different language groups are treated”. Yet “what you’re being pushed to do by both clients and investors is claim that you are… producing language technology for all African languages and even scaling globally. Like it’s all about scale”.

📚 Deeper theory

  • The “announcement effect.” A big-lab press release can change investor beliefs even before quality is proven. Claims like “200 languages” describe coverage, not quality in each language. Benchmarks for the lowest-resource languages are thin. (General point; I did not verify NLLB’s per-language scores.)
  • The data bargain. Selling data to a lab “like peanuts” is a form of extraction (idea 3). The data’s value is high to the lab and the group has little bargaining power. Te Hiku’s refusal is the counter-example.
  • Context is part of correctness. A translation of a political phrase in Ethiopia depends on who is speaking about whom. This is what she means by “contextual knowledge”. A model trained on web text lacks it, and evaluation by outsiders may not notice.
  • Why investors reward scale. Venture funding looks for markets that can grow ten-fold. A community-owned language service may be a good, durable business and a poor venture bet. That is a mismatch of funding model, not of merit.
  • Her sourcing. These are second-hand accounts from conversations with two groups. She does not name the investors or labs. The pattern is plausible; the specific claims (threats, “peanuts”) are not independently confirmed.

📚 Roots: Low-resource languages and language technology

The problem. Most of the world’s languages have little text or audio online, so mainstream models serve them badly.

The point. Her key claim: context knowledge (“how different language groups are treated”,) is data quality, and only locals have it.

  1. Joshi, Santy, Budhiraja, Bali and Choudhury (ACL 2020), “The State and Fate of Linguistic Diversity and Inclusion in the NLP World”. Showed that a small number of languages receive nearly all research attention. (From memory.)
  2. Emily Bender’s “#BenderRule” (2019). Name the language you work on, because English is not “language in general”. (From memory.)
  3. Masakhane (2019–). A grassroots African research network for machine translation and NLP, using participatory research: speakers build the data and models. Nekoto et al. (2020) is the key paper. (From memory; my search found no Masakhane results.)
  4. Meta’s NLLB (announced July 2022; Nature paper June 2024). 200 languages, with a cross-language transfer approach that lets low-resource languages learn from high-resource ones (coverage, multilingual.com). Roughly three-quarters of the 200 are low-resource. I could not confirm her “fifty-five African languages” figure independently; the AI Now transcript has the same wording.
  5. Lesan, Ghana NLP, Lelapa AI. Small organisations with local context. Lelapa’s InkubaLM (0.4B parameters, five languages) shows a small model can be trained from scratch for African languages (Lelapa).
  6. India as a parallel case. AI4Bharat’s IndicTrans2 and the government’s Bhashini programme are also multi-model efforts for many languages. (From memory; relevant to the India Summit framing; not checked.)
  7. Why general models tend to do worse here. Training data is dominated by English and a few languages. Quality is tied to the amount and cleanliness of data. A Fast Company piece reports that smaller models trained on one language often beat large multilingual ones on non-dominant languages (Fast Company). The claim is often, not always.

Where others push back.

  • Meta released NLLB’s models openly, and many small groups use them as a starting point. For low-resource languages, transfer from larger languages is a real technical gain. Some small groups welcome this.
  • Investors may be right on market grounds: if a general model reaches “good enough” quality, a small provider loses its edge on price.
  • Customers asking for broad coverage are asking for a real need: a business with users in 20 countries wants one vendor.

What she leaves out. Evidence on whether the general models are in fact good enough in Amharic or Twi. If they were bad, the small groups’ value would be obvious to buyers. If they were good, her case rests more on ownership and ethics than on quality. She mixes the two.

Check yourself

  1. What is the difference between coverage and quality, and why does it matter for NLLB?
  2. Why might a venture investor and a community language group want different things?
  3. What would count as independent evidence for her account?

7. The federation idea and “pluralism”

In short. The host calls small networks sharing resources a form of pluralism. Gebru’s proposal is a federation of small organizations. They would share data among themselves, and outsiders could reach all of them through one interface. Together they could compete with the big players, and each curates its data well. She presents this as her idea, not a result.

🎙 Host. “This kind of small network building around resources is a form of pluralism. And I think hegemony hates pluralism… This isn’t an influence operation. This is a let’s do things the way we want”.

🎙 Gebru. “My idea was to have a federation of small organizations where they share data amongst themselves, but then externally there’s an easier interaction between them. So if you want language support in both Ghanaian languages and Ethiopian languages, you can interface with one application interface”. “Perhaps they could also band together and present themselves as a really good competition to these bigger organizations. And at the same time, each of them knows their context so well and they curate data so well. They’re not just guzzling everything on the internet. They’re really curating data and using low resource models”.

📚 Deeper theory

  • Federation as a design pattern. Think of a co-operative bank network, a union of dairy co-operatives (India’s Amul model is a well-known case), or the internet itself: separate parties, shared protocol. Each keeps its own data and rules; a thin layer joins them. (Analogy mine; she does not mention Amul.)
  • Technical shape. One API that routes a request to the right member model, and shared agreements on data format, quality and pricing. The hard parts are trust (who sees the shared data?), quality control (who checks each member?) and money (how are revenues split?).
  • Data pooling vs data sovereignty tension. She proposes both: groups “share data amongst themselves”, and Te Hiku refuses to give data away. These fit only if sharing is within the federation under rules (like the Kaitiakitanga licence) and not with outsiders. This tension is not discussed in the talk.
  • Pluralism. In political theory, a plural order has many centres of power. Arturo Escobar’s “pluriverse” (2018) is a related idea in design: many worlds, not one. (From memory.) The host uses the word; Gebru does not.
  • Curated data. Curation is the quality argument for small. A smaller, well-checked set can beat a larger noisy one for a narrow task. This is a well-known pattern in machine learning, though it depends on the task.

Where others push back.

  • Coordination costs. Federations often struggle. Members have different incentives, and a central body may become the new gatekeeper. Many open-source and co-operative projects stall on governance.
  • Scale economies in the product. Customers want fast, cheap, always-on APIs with support. A volunteer-run federation must match this, or it will lose to the giants on service even if it wins on quality.
  • Scale also helps: shared multilingual training gives low-resource languages a boost from related languages (NLLB’s own finding). A federation that keeps data separate could lose this transfer.

What she leaves out. Governance, money, and technical standards. Also whether any such federation exists yet. In the talk it is “my idea”, a proposal, not a result.

Check yourself

  1. Why does data sharing inside a federation not clash with Te Hiku’s refusal to give data to a US firm?
  2. What are the three hardest problems a federation must solve?
  3. What does shared multilingual training give that separate small models might lose?

8. “Secular religion”, DeepSeek and the hijacked imagination

In short. Gebru calls the drive to build a machine God a “secular religion”, pushed by billionaires for two decades. A task-specific tool is ordinary engineering. What is new and ridiculous is the giant model. She says DeepSeek innovated under constraint, but it still followed the large language model paradigm. She asks why everyone’s imagination is hijacked, and wants help to free it.

🎙 In the talk. “If you don’t have the resources on earth, let’s go to Mars and let’s go mine some, I don’t know, outer space. That’s what these people are thinking”. “I would call it like a secular religion to try to create a machine God that has been pushed in the last two decades by various billionaires putting in so much money and funding”. “To me this idea of a task-specific tool is just a concept in anything we build in engineering. It’s not really a new concept… And it’s not the frugal AI thing that you’re talking about that’s new. It’s more the flip side of what we’ve been seeing that’s ridiculous and new”. On DeepSeek: “when you are constrained, when you can’t get the best GPUs because of export controls, that’s when you innovate… So what’s happening in the US is not innovation. On the other hand, DeepSeek was still this large language model kind of paradigm. And I am thinking, why is everybody’s imagination hijacked?“. She ends: “We really need to produce content or something to help people unhijack their imagination into something different”.

📚 Deeper theory

  • The religion claim is the TESCREAL argument (Roots). It is a claim about motive and ideology. It does not by itself show the technology is poor, and critics say it is hard to test. Note the strong word “stealing” and “ridiculous”: these are her judgements.
  • DeepSeek facts. V3’s reported cost of about $5.6 million was disputed as a partial figure. R1’s own cost was later given as $294,000 on 512 H800s in a Nature paper, with earlier A100 use admitted (Al Jazeera, BNN Bloomberg). The “constraint breeds innovation” story is common but contested: US officials said DeepSeek had access to many H100 chips, which Nvidia disputed.
  • Her two feelings map the argument: cleverness under limits shows the scale route is not required, yet the clever people still built an LLM. This suggests the pull of the paradigm (funding, prestige, benchmarks, jobs) is strong everywhere.
  • Jevons paradox. When a resource becomes cheaper to use, total use can rise. Cheaper LLMs could mean more LLM use and more total energy. Several commentators raised this after DeepSeek (general point)†. It means efficiency alone does not cut total cost.
  • “Unhijack the imagination.” A call for narrative work: stories, examples, demos of other designs. This is the role of case studies like Te Hiku. It is also why she answers a policy question with a communication task.

📚 Roots: Appropriate technology and “small is beautiful”

The problem. Should technology always grow in scale, or be fitted to the people who use it?

The point. She places herself in the old tradition and calls the giant model the new oddity. That is a rhetorical choice you should notice: it moves the burden of proof onto scale.

  1. E. F. Schumacher, Small Is Beautiful (1973). “Intermediate technology”: tools sized to local skills, money and resources, so that communities can own and repair them.
  2. Ivan Illich, Tools for Conviviality (1973). A tool is “convivial” if people can use it freely and shape it. Past a certain scale, tools control their users.
  3. Langdon Winner, “Do Artifacts Have Politics?” (1980). Some technologies suit centralised control by their very design. Key point: this supports the host’s reading that universal tools allow concentration of power.
  4. Frugal innovation (India and elsewhere, 2000s–10s). Low-cost products for low-income markets. Gebru does not use this history. The host’s word “frugal” belongs here, and she says the idea is old: “it’s not the frugal AI thing that you’re talking about that’s new”.

📚 Roots: The ideology of “AGI” — the “secular religion”

The problem. Why would anyone build one model for everything, at any cost? Her answer: a belief.

The point. She ties the technical choice (giant model) to the belief (machine god), which is why she says small groups are “not trying to build the one machine God that rules them all”.

  1. Nick Bostrom, Superintelligence (2014). Popularised the idea that a machine smarter than people could be an existential risk or a saviour. (From memory.)
  2. Effective altruism and longtermism. Movements that give heavy weight to far-future outcomes, and funded much AI-safety work. (General background.)
  3. Gebru and Émile Torres, “The TESCREAL bundle” (First Monday, 2024). Argue that a set of linked ideologies (transhumanism, extropianism, singularitarianism, cosmism, rationalism, effective altruism, longtermism) shapes the AGI race and echoes eugenics. (†) Key point: “secular religion” is a short form of this paper’s thesis.
  4. The Mars and outer space remark is her mockery of a resource logic with no limit: if Earth cannot supply it, go and mine space. It is a joke, not an argument from data.

Where others push back.

  • The “religion” frame can dismiss real technical progress and real safety worries by pointing at motive. Many AI researchers who are not billionaires’ employees hold that more general systems are possible and worth building. Arguing about beliefs does not settle what the systems can do.
  • Empirical record. Between 2020 and 2026, general models gained many abilities (code, maths, translation, reasoning) that small narrow systems did not. If this trend holds, the Bitter Lesson view looks stronger, not weaker (idea 1).
  • Not either/or. Most deployments use both: a large model to produce training data or to teach small ones (distillation), and small models to serve. The real design space is a mix.

What she leaves out. A positive account of what a good general system would be, if any, and how to tell it from the bad kind. She also does not say where her line falls: is a 7-billion-parameter chat model “small” or part of the problem?

Check yourself

  1. What are her two feelings about DeepSeek, and what do they show together?
  2. Why might cheaper AI raise total energy use (Jevons)?
  3. Which of her claims in this section is about motive, and which is about technology?

Q&A

(Add questions and answers here only if worth keeping.)