An Untangled System: Using AI as a Teacher

AI as a Teacher: How to Use AI Tools to Learn About New Systems

I have a set of AI tools I use regularly as part of my daily workflow. In this post, I’ll share how I use them to learn about a system I was unfamiliar with, and how these tools have become part of how I teach myself new things.

I have been thinking a lot about how games can help people learn to use AI. I have been building a game called Promptatorium where you write prompts to create organisms that then go live in worlds with other prompt created organisms.

I was listening to the podcast AI & I and I learned about using games to evaluate LLMs themselves, and I had to dig in.

On AI & I, Dan Shipper was interviewing Alex Duffy about how they built a system where LLMs could play the strategy game Diplomacy. If you are not familiar, Diplomacy is a game where you play as a country competing for power. Good Start Labs used it as a sandbox for LLMs to compete and collaborate.

By having LLMs play games, they found a new way to evaluate models by seeing how they did against each other in a game. Since I have been exploring how games can teach people about LLMs, this approach caught my attention. I began to utilize my favorite set of AI tools to understand more.

The first thing I did was read the post about their system. The code was open source, so then I forked it on GitHub and cloned it locally. Using Claude Code, I generated a CLAUDE.md file to help me ask questions about the repo. I also checked Deep Wiki to see if AI-generated documentation existed, and it did.

Deep Wiki is a project from Cognition, makers of the AI coding agent Devin which I tried earlier in the year. It never became my go-to collaborator, but its automated documentation is excellent. You can use it for your own private repos with a subscription, but it also offers public documentation for open-source projects.

The high-level diagrams on Deep Wiki did not immediately click for me, so I dove in through Claude Code. I started with:

“Can you describe all the major parts of the game to me?”

The first answer was too detailed, so I followed up with:

“Can you describe it at a higher level?”

That was perfect for where I was in my learning. It mentioned that each agent has an internal diary, which intrigued me. I asked Claude:

“Can you point me to where the diary is in the code and explain how it works?”

This led me to the Agent architecture and revealed that each agent keeps two diaries, one full version and one optimized for the LLM itself, as well as a relationship map tracking trust and alliances between players.

I kept exploring, trying to understand what I could bring back to Promptatorium, the life-simulation game I have been building where an LLM helps you create a creature from your description.

I opened my game’s repo, activated Claude Code, and added the AI_Diplomacy directory using the /add-dir command. Then I created a sub-agent with the prompt:

“You are an expert gameplay evaluator. Your expertise is in comparing methodologies between games and suggesting improvements.”

I asked it:

“Using the gameplay evaluator, can you compare Promptatorium to AI Diplomacy and share what you notice?”

The two games are, unsurprisingly, quite different, but the contrasts were fascinating.

Claude Code’s Analysis

Core Design Philosophy Differences

Promptatorium: Fast iteration, emergent behavior, token efficiency

• 5-minute episodes with immediate LLM feedback
• Context compression (150–200 tokens per decision)
• Deterministic replay via seeded RNG
• Weekly leaderboards for fresh competition

AI Diplomacy: Deep strategy, rich context, long-form engagement

• Multi-hour games with extensive negotiation
• Full memory systems (goals, relationships, diaries)
• Async agent orchestration for seven simultaneous powers
• Post-game analytics and betrayal detection

Claude Code’s Recommendations

Immediate Recommendations for Promptatorium

High-impact, easier implementations

• Episode tournaments with special rewards
• Organism dynasties (trait inheritance)
• Asymmetric maps with distinct biomes

High-impact, complex implementations

• 30-second negotiation phase before episodes
• Memory fragments (organisms remember key threats and allies)
• Trust scoring system

I had already been thinking about how my game always takes place on a single planet, so “asymmetric maps with distinct biomes” immediately stood out. I asked Claude Code to create an implementation plan. In plan mode, it now sometimes asks follow-up questions, which makes the process even more interactive.

I am still improving Promptatorium before I release it into the wild, but exploring AI Diplomacy reminded me that the best way to learn about AI is not just by reading. It is by playing, building, and experimenting with the agents themselves.
​
Are you using AI to learn? What tools or prompts work for you? Would you like me to dig into any of the tools I used here more in depth?

-Kate