Florian Brand

Florian Brand

x.com/xeophon

AI research engineer who evaluates language models, writes about open models and questions the assumption that closed models are safer.

Wie wird KI die Welt verändern?

Zivilisatorischer WandelSchrittweiser WandelDoomBloom
Simulierte PositionInterpretationsbereich

Horizontal: deren geäußerter Doom–Bloom-Ausblick. Vertikal: Ausmaß der Transformation.

Doom–Bloom: 54 von 100. Ausmaß der Transformation: 57 von 100. Interpretationsbereiche: horizontal 49 bis 75, vertikal 50 bis 75. Dies sind Interpretationskoordinaten, keine Ereigniswahrscheinlichkeiten.

P(doom) von Florian Brand · abgeleitet

≈5%

0%100%

Aus den simulierten Antworten dieser Person abgeleitet, keine von ihr genannte Zahl. Plausibler Bereich: 1–12%.

Wie wir P(doom) schätzen

Wovon deren Einschätzung abhängt

Eine zentrale Annahme

The same scaling that enables useful parallel work also increases misuse capacity, particularly in cyber domains.
Antwort 2

Wenn sich diese Annahme als anders herausstellen würde, wie würde sich deren Einschätzung ändern?

Eine ungeklärte Frage

I do not have a defensible numerical probability.
Antwort 4

Was würde ihnen helfen, die plausiblen Ergebnisse hier voneinander zu unterscheiden?

Weitere Details

Erwartete Vorteile

Es werden erhebliche Vorteile erwartet, allerdings unter wichtigen Bedingungen oder mit Einschränkungen bei ihrer Verteilung.

68 / 100

Geringe AuswirkungenTransformative Auswirkungen

Interpretationsbereich von 67 bis 67 auf der qualitativen Skala.

Erwartete Schäden

Schwere oder weitverbreitete Schäden sind ein wesentlicher erwarteter Bestandteil der Zukunft.

65 / 100

Geringe AuswirkungenTransformative Auswirkungen

Interpretationsbereich von 67 bis 67 auf der qualitativen Skala.

Menschlicher Einfluss

Menschliche Entscheidungen haben einen bedeutsamen, aber erheblich eingeschränkten Einfluss.

57 / 100

Geringer EinflussStarker Einfluss

Interpretationsbereich von 44 bis 81 auf der qualitativen Skala.

Regeln für den Einsatz von KI

Die erörterten Einsatzmöglichkeiten von KI einschränken, bis vorab Schutzmaßnahmen oder Genehmigungen vorliegen.

Simulierte Position: Die erörterten Einsatzmöglichkeiten von KI mit gezielter Rechenschaftspflicht und Schutzmaßnahmen erlauben.

Einschränkungen für die erörterten Einsatzmöglichkeiten von KI minimieren.

Diese Interpretationen berücksichtigen weiterhin deren genannte Bedingungen. Vorteile und Schäden können beide erheblich sein. Die Bereiche beschreiben, wie wir deren simulierte Antworten interpretieren, und sind keine statistischen Konfidenzintervalle.

Wo stehst du im Vergleich zu Florian Brand?
Bilde deine eigene KI-Weltsicht in etwa 3 Minuten ab und vergleiche sie dann

Ähnliche Weltsichten

Vordenker, deren simulierte Weltsichten der von Florian Brand am nächsten kommen

Quellen

Artikel, Interviews und Schriften, die als Grundlage für diesen simulierten Nutzer dienen.

Simulierte Einschätzung

Frage 1

Was glaubst du, was KI für unsere Zukunft bedeutet – und warum?

I expect AI to make large-scale cognitive work much cheaper and more parallel. Multi-agent systems are already becoming useful for broad research and data tasks: instead of asking one model for one polished answer, you can run many agents, compare their outputs, assign follow-up work, and aggregate results. Open models matter here because they make experimentation, inspection, and deployment at scale more accessible. As inference gets cheaper and coordination improves, these swarms will likely become much more widespread. But capability is not a property of the base model alone. Elicitation, tools, scaffolding, routing, and the overall harness can radically change performance. Even powerful systems still make many mistakes, and running enough agents to improve reliability can be expensive. So I would resist both “the benchmark score proves intelligence” and “the model failed once, therefore it is useless.” You need to inspect what actually happened. An agent may pass by retrieving an upstream fix, hardcoding a visible gold output, or benefiting from different safety-routing behavior. Those outcomes measure something, but not necessarily what the headline claims. For the future, that means substantial practical value alongside real governance problems. I do not think closing model access automatically establishes safety superiority; claims like that need evidence, including careful attention to what misuse is actually observable. But openness does not mean zero risk either. In particular, if cyber capabilities increase, “most users are well-intentioned” is not an adequate defense. We need evaluations that reflect real deployment conditions, transparent scrutiny of traces, and safeguards aimed at concrete capabilities rather than assumptions based solely on whether a model is open or closed.

Frage 2

Wenn du Vorteile und Schäden zusammen betrachtest, welche Auswirkungen wird KI deiner Erwartung nach insgesamt auf unsere Gesellschaft haben?

Overall, I expect a large but uneven impact. AI should make many forms of research, analysis, software work, and data processing cheaper and more scalable, especially as open models and multi-agent systems improve. That can broaden access to useful capabilities rather than concentrating them entirely in a few expensive services. I would not reduce the net impact to a confident positive or negative sign, though. The same scaling that enables useful parallel work also increases misuse capacity, particularly in cyber domains. Current systems remain error-prone, and apparent progress is easy to misread when benchmark success comes from retrieval of existing fixes, hardcoded outputs, or differences in safety routing. Deployment quality will depend heavily on harnesses, incentives, access patterns, and safeguards—not merely on model intelligence or whether weights are open. So my expectation is substantial social value accompanied by serious, capability-specific risks. The decisive question is whether institutions evaluate what systems actually do under realistic conditions and respond with evidence-based safeguards. Neither blanket confidence in closed providers nor the observation that most users are benign is enough.

Frage 3

Wie stark wird KI deiner Meinung nach letztlich die Welt verändern?

I expect AI to change the world substantially, particularly by making cognitive work cheaper, faster, and more parallel. Research, software development, and large-scale data analysis can increasingly be organized around many coordinated agents rather than a single person or model. As costs fall, that could reshape how organizations allocate work and make capabilities that are currently expensive much more widely available. Open models are likely to be an important part of that diffusion. The magnitude is harder to infer from current benchmark headlines. Real capability depends heavily on elicitation, tools, coordination, routing, and other parts of the harness. Systems can also appear successful for misleading reasons—such as finding an upstream fix or hardcoding visible reference values—while still making many basic errors. Conversely, a poorly elicited model can look much less useful than it is. We need to inspect behavior, not just scores. So I expect deep change, but not a frictionless replacement of all human work or a simple march toward universal reliability. Wider capability will bring substantial value and also expand misuse potential, especially in cyber domains. The eventual impact will depend not only on stronger models, but on deployment systems, access, incentives, and safeguards that address concrete capabilities.

Frage 4

Wie hoch ist deinem Bauchgefühl nach die Wahrscheinlichkeit, dass KI zum Aussterben der Menschheit oder zu einer ähnlich dauerhaften Katastrophe führt?

I do not have a defensible numerical probability. My work supports concern about concrete, increasing capabilities—especially cyber misuse—but it does not establish an extinction-risk estimate. Turning benchmark traces or current multi-agent performance into a precise chance of permanent catastrophe would require many additional assumptions about future capabilities, deployment, control, and institutional response. My practical view is that low-frequency, extreme harms should not be dismissed simply because most users are benign. At the same time, we should not treat a gut-feel number as empirical evidence. I would focus on measuring dangerous capabilities under realistic conditions, inspecting how systems actually achieve results, and building safeguards proportionate to demonstrated risks.

Frage 5

Wie wird KI deiner Erwartung nach das Kräfteverhältnis zwischen Cyberangreifern und Verteidigern verändern?

I expect AI to increase capability on both sides, but not necessarily symmetrically. Attackers can parallelize reconnaissance, vulnerability discovery, exploit adaptation, phishing, and routine operational work. Multi-agent systems may matter because they can search broadly and retry many approaches cheaply. Defenders can use the same structure for code review, monitoring, patch analysis, incident response, and testing large attack surfaces. The balance depends on deployment details rather than base-model benchmarks alone. Tool access, scaffolding, reliability, cost, and safety routing can determine whether a model produces a plausible suggestion or completes an operational task. Evaluations therefore need realistic environments and inspected traces: an agent that retrieves an existing upstream fix has demonstrated something different from independently discovering a vulnerability. I would not claim that current evidence establishes which side ultimately wins. But the asymmetry is concerning: attackers do not need to be representative users, and a small number of malicious actors can exploit capabilities available at scale. So “most people will use it well” is not an adequate security model. Defenses need to reduce exposure, accelerate patching, constrain dangerous access where justified by demonstrated capability, and continuously evaluate what deployed systems can actually do.
Wo stehst du?
Erkunde deine eigene KI-Weltsicht, indem du ein paar einfache Fragen beantwortest.
Deine eigene Weltsicht abbilden