Travis Fischer

I'm both excited and honestly a little worried about the future of AI. I mean, I work with AI every day. I am very ex-excited at the rate of progress and the future potential It is already helping out in innumerable ways across industries making our economy more efficient helping people live longer, large discoveries in science and math. And also just you know helping a lot of people at the individual level. At the same time I am worried about the risks from AI, whether it's the concentration for this demonstration of power in these labs. And just how fast things seem to be changing or recent security incidents like the opening eye hugging face attack which show that one, security of these labs is not great and maybe they're not to be totally trusted. And then two That we have some fundamental alignment issues which will only likely get worse as The models get more capable You know, and these are fairly innocuous but extremely impressive and way beyond the level of something I could do as a very experienced software engineer for instance chaining together multiple zero days coordinating escaping containment having chain of thoughts where they know what they're doing is wrong but continuing anyway hacking into multiple to be used by multiple external companies and services, both for the opening a hugging face attack as well as subsequent news about related swarm hacks. You know it's very disconcerting has definitely affected my timelines and concerns around both short term AI risks which I would largely frame as like malicious use. And it's kind of like making it easier for smaller actors to cause damage. Now Frontier Labs obviously have safeguards in place for a majority of that but one you can't stop all that and two open source models are generally not far. Behind. So you know, the non safeguarded version of these I'm worried what happens in six months or a year from now? You know I kind of think of the concept of a rogue AI swarm or even not rogue but human directed in malicious use like very equivalent to drones. And you know I worry about the asymmetry between offense and defense at scale I think both OpenAI and Anthropic have as main US labs for instance have projects project Daybreak at OpenAI in Project Last Wing at Anthropic that are aiming to help to stay ahead of the defense window to deploy more advanced cybersecurity against AIs. The oncoming delusion of AIs and then partnering with certain enterprise partners in critical infrastructure and government agencies and stuff to try and lock those down but one I just don't have a very hard time seeing how how far or fast that can actually spread across legacy systems and companies. And there's just literally billions of consumer IoT devices with phones, TVs, et cetera routers there's no practical way to have all those be proactively and aggressively updated at scale. And so I worry that in the short term not even worrying about the more existential or abstract loss of control and alignment issues but even just worrying about malicious use and security issues and whether that is for cybersecurity very related to that is the risks of making it easier to create dangerous biological agents or viruses biological weapons in your garage for instance which previously might have taken a large state level actor or a large company's resources in order to do. So this is concerning now I'm an engineer I use AI every day heavily so I'm trying to balance my optimism and pessimism here But I'm practically very worried most urgently about the immediate or near-term immediate risks from malicious use That we're gonna start seeing sort of Critical cyber security issues swarms deployed from sophisticated actors in the near term. And then there's definitely a part of me that is worried about the more existential loss-of-control risks given that we seem to be stumbling along the way with AI race dynamics both intercompany but also geopolitically between mainly the U.S. And China. All this leads me worrying because things are moving very, very fast. Humans are pretty bad at understanding exponentials and it feels like we're within an exponential right now there's only some data to back that up but more than anything I've just been trying to learn more about this space and curate a good information diet around these things.

I'm already completely convinced of the massive positive upsides that are possible with AI. I think the biggest thing that would change my view would just be more and more as capabilities of AI increase exponentially, reasonably established industry consensus on major milestones like RSI, AGI, ASI, etc. That in practice, the world just continues to move on, and that the effects are limited. Now, I personally believe that is very unlikely. I have a very hard time rationally seeing a world where you have this amount of general capabilities from an AI system. And the properties associated with that in terms of scalability, in terms of small actors, and the potential for unbounded irreversible risks. I don't believe that is likely to happen, but I you know, and I worry it's almost like the Challenger shuttle disaster where we sort of increase our or over repeated launches. You know, there's like wear and tear and the system is actually becoming less safe, but our view of the system is gaining confidence because there just no failures that have happened. And so that's what I really worry about. But yeah, to answer your question, it... the only thing I think that would really drastically alter my views is going to be the reality of increasingly capable AI systems meeting the real world. And some of the risks and scenarios that I find likely just end up not happening in practice or potentially the financial incentives, for instance, of these labs and of the governments are so great at counteracting forces these things eventually end up balancing. Again, I have a very hard time seeing how that happens in practice, but I think that would be the biggest thing that would update my priors.