Towards an AI Research Agenda for Elections and Beyond (Part 1)
Courtesy Getty Images
I recently posted an endorsement and amplification of Judy Estrin’s key point in her article: “The Case Against AI Everything, Everywhere, All at Once” that the “tech titan” companies are driving the research agenda for AI technology, to nobody’s benefit but their own. In the next couple of posts starting here, I want to build-out on that point a bit, and connect it to the AI needs both in election technology and in government technology more generally.
Going forward, I will limit my remarks to text-based generative AI, the technology behind “chat-bots” and other kinds of natural language agents (NLAs). There is of course plenty to say about AI more broadly, but NL AI is the technology that can meet important needs in human assistance, specifically NLAs that are “domain specific”, that is,
developed specifically to be limited to one domain of human knowledge that is largely represented as textual information (for example, this); and
developed with the sole purpose of helping people navigate that specific knowledge base, which is too large for a single person to master without a major time investment.
Lots of Folks?
My first point about the research agenda is to preemptively rebut the objection that lots of other people are doing NL AI research.
Sure, academics are doing basic research in NL AI, but applied research is gated by one big thing: big money. It takes an enormous (and expensive) amount of computing cycles/time (aka “workloads”) to build a new large-language-model (LLM) that can serve as the base model for NLAs. Academics may be working on the science of next-generation NL AI, but applied research on the current generation of NL AI is essentially limited to those with the financial capacity and organizational resources to build new base models.
“Sure,” you say, “But aren’t lots of people doing research on open-source AI?” No: and the key word is “on”. Lots of organizations are using readily available (including available free of charge) pre-built base models. And some may be using them to perform “research” in the sense of experimental use of these base models to try to develop something for their organization. So, while that might be research using natural language (NL) AI, it is not research on NL AI itself. It’s not research on how to build better base models that can be trained more effectively and better constrained to avoid the generation of hallucinations. Somewhere on the bleeding edge of the experimental usage is what I call “band-aid” research: how to use existing fundamentally flawed base models employing band-aid techniques to reduce very adverse consequences.
NOTE: If you detect a theme around “band-aid”, you’re right. I often talk about band-aid defenses of fundamentally flawed voting systems: they’re inherently vulnerable to cyber attack, so plaster them with band-aids of air-gaps, procedural controls, physical security, personnel security, etc.; and then tell stories about how every one of 1000s of local jurisdictions do the band-aiding flawlessly with no process audits.
Any research beyond “band-aid-ology,” at scale, is pretty much the domain of the tech-titans, although I suspect that a lot of their work is also band-aid-ology. Either way, the actual research on NL AI tech itself is being performed by the titans who have the resources to do it; and that research is primarily to advance business objectives, not public benefit.
“Open-Source” AI?
A second point is to disentangle the extremely regrettable phrase “open-source AI.” “Open-source” has a specific meaning in “open-source software” and that meaning does not apply to AI. The NL AI things that you can obtain without payment might be “open” in some meaning of the word, but you’re not gaining access the source code to a system that you can do your own software build of, and modify as you please, based on a full understanding of how it’s built, and of how it works. Instead, you’re getting a black box that you can tinker with around the edges to customize it somewhat. I won’t say more because Heather Meeker, one of the world’s pre-eminent experts on open source software licensing (and the OSET Institute’s OSS legal counsel), gives a full and well explained story in a short video (which we heartily encourage you to spend the 5-minutes to watch).
Let’s give Heather her point, it’s not “open-source,” but there might be stuff loosely called “open” AI tech (not to be confused with the company Open AI, which makes proprietary closed systems (we know; it can all be so confusing 🙄)) because it's easy to get a copy of a non-proprietary base model, customize it a bit, and build your own NL agent. Sure, but your agent arrives, warts and all, from the base model’s polluted base data, biases, and impressive hallucinogenic abilities. And in fact, most organizations don’t even get their own copy, they just get the use of an existing system via API access to models running in the cloud. That’s not “open” anything, it's just leveraging an API to use yet another black box; whether or not you pay for the privilege … doesn’t make it “open.”
To me, these “open” AIs are nothing like “open” anything, because there is no transparency of these admittedly freely available things that I can use to make a system of my own … which will have mission-critical failures when it lies and hallucinates. If you want a complete dissection of how “open” doesn’t really fit “AI”, your go-to source is an academic paper “Open (For Business): Big Tech, Concentrated Power, and The Political Economy of Open AI” by David Gray Widder, Meredith Whittaker, and Sarah Myers West. They give an excellent account of a spectrum of “open-ish-ness” of AI tools. The only thing on the spectrum that feels close to “open” similar in spirit to “open source software” is work being done by (no surprise!) a non-profit organization, Eleuther-AI, to provide tools to NL AI researchers who have the financial and organizational capacity to try to do some work on their own.
What’s Needed … Not Profitable
Even so, is that research the basis for the kind of NL AI research that I am talking about? No, it’s not the research toward domain specific NL agents that I described at the top of this post. That’s because in many important cases, domain specific NL agents cannot be used to make a bunch of money. To go back to my earlier example, it would be wonderful to have an advice-bot to help people dealing with eating disorders, and that does not provide harmful advice! Yet, given the considerable work and cost required for trying to make such a thing, I doubt the work would be done by a for-profit company expecting to profit from it.
And I suspect the same thing about urgent needs in election-land, which I will turn to next time.
