LLMs in Hacking Competitions (CTFs)

I’m just writing this post to express my brief thoughts on AI being used in CTFs and the CTF community in general right now. Obviously, this is just my opinion, and you don’t have to agree with me.

Overall Thoughts

Around the middle of 2025, LLM CTF solvers started to become a problem, especially with the release of Codex and Claude Code. Teams with no cybersecurity experience could just pull up an LLM and solve challenges in several prominent CTFs. This became an even more drastic problem by the end of 2025, as newer versions of the aforementioned agentic tools were released. The CTF community did not respond well to this at all; players like krauq who won several CTFs just using AI were receiving enormous amounts of hate on Twitter and other platforms (he eventually quit CTFs due to the backlash). I think this is a classic “don’t hate the player hate the game” moment. If challenges can be solved with AI, then not solving them with AI poses a competitive disadvantage. One good example of this was picoCTF 2026 (a beginner CTF), comparing the 2026 leaderboard to the 2025 leaderboard, the discrepancy is insane. Tons of people completed every challenge because AI was able to solve them all, with all the top teams using more efficient AI agents to gain an edge.

That being said, the rise of AI in CTFs is a detriment to the fun and learning provided by challenges. Competing in “slop wars” is not fun for anyone, and since CTFs are games at heart, “fun” is one of the main aspects that make people play them. That means the game has to evolve or die, and by evolve I do not mean ban AI. In the modern cybersecurity world, the people who are able to integrate AI into their workflow are the ones who will succeed. Trying to ban it does not reflect the current state of cybersecurity.

The CTF Community

Ohhh boy… The CTF community definitely has some toxicity issues because of the rise of AI, which I have witnessed first hand. People seem to tell themselves that anyone who is using AI is unskilled, which in most cases is frankly untrue. I know some of the best players use AI, people I look up to. In my opinion, everyone should remember, CTFs are games, just that. Attacking other people for using AI is just not necessary or warranted, unless it is directly against the rules of a certain CTF. We are all working together in the cybersecurity community to keep people safer and educate others on the importance of security. I get that it can be pretty annoying to see teams who might be less skilled than others at the top of the CTFtime leaderboards, but that’s just a product of the current CTF landscape. Not to mention, CTFtime rankings more accurately tell people how active a CTF team is rather than how skilled a CTF team is.

The people that really need to reflect are the ones who publicly attack others using AI, but then go back and use it themselves whenever they deem fit. Yes, maybe you never wanted to use AI and are only using it because others are using it, but that’s just what the landscape is right now. Constantly berating others about it is not going to help anything; instead, helping to figure out how CTFs can evolve to naturally make these teams less successful and keep the game fun is a much better use of time.

Banning AI

Now, back to my earlier point about CTF evolution, this cannot happen through banning AI for two main reasons. The first reason is that it’s just not practical to verify if people are playing with or without AI in online competitions. The only way I would really see this happening is if video footage is submitted for all solved challenges, but that is a privacy invasion that most people won’t submit to. There will always be suspicion of teams using AI even with this measure to be honest, as there are always ways to trick the system. Previous CTFs that have banned AI have mostly ended in disaster, with several of the top teams getting banned even though they stayed within the rules of the competition. Many of these rules were vague and overall the CTFs were just not a good experience. Most bans were based on vibes and preconceived notions rather than hard proof. The second reason is that it is just not helpful. AI can do cybersecurity whether we like it or not. Not adapting to it is detrimental for overall progress in the field. CTFs have to adapt to cultivate cybersecurity skills that AI is still not good at. Those will be the skills demanded in the workforce, along with being able to use AI effectively to speed up vulnerability research and problem solving. There is also a chance that it may not be possible to adapt the CTF format, in which case, CTFs will have to turn into a purely learning experience with no prizes. There is no incentive to spend tokens for no reward.

Learning

The one part I definitely agree with the AI critics on is that it is a detriment for learning. If you have no cybersecurity skills at all and are winning CTFs, yes, that might look good on a resume, but you won’t actually have any skills that might still be necessary in a cybersecurity job. CTFs used to be a great way to learn and compete, but now, a single goal has to be picked. If you want to learn, CTFs should be done with no AI and no focus on the leaderboard whatsoever, and if you are trying to compete, AI probably has to be used. Humans just can’t solve challenges faster than AI in most cases. Outside of CTFs, platforms like pwn.college are absolutely incredible for learning because they don’t focus on competition. I can’t tell you how much I love the pwn.college platform; it has some awesome challenges that have really helped me understand both basic and complicated topics in various disciplines of cybersecurity. Huge shoutout to all the legendary people at ASU for creating the challenges.

Final Thoughts

As of now, I have quit competitive CTFs for the most part. DEFCON Finals is the last competitive CTF I will likely play with the amazing people @Shellphish (unless CTFs adapt), and I will be treating it mainly as a learning experience. I am not going to lie, DEFCON is above my level of raw skill (as it is for almost every high schooler not using AI; basically all high schoolers who qualify for DEFCON are relying on more experienced teammates in larger teams to learn from), but it will be really fun helping develop AI agents for the A/D portion of the CTF and learning from experts at the finals in August! Thanks for reading my post, and feel free to email me your opinion. I am very curious to see the different takes on this issue (contact@milrn.dev).