How I tested a fly’s brain in minecraft using only my laptop
I managed to get a living fly’s brain inside of my computer, not only that, but I then managed to put it into the most played game in the world, Minecraft.
I didn’t actually put a fly below my keyboard, but I did copy its brain. This was able due to the connectome being opensourced by flywire and available for download.
I replicated a fly’s brain to play minecraft. Using the flywire connectome and existing mods for compatibility, I got a fly that actively learns in the virtual world. The simulation shows how all of her neurons are used to proccess different functions. The biggest thing I learned was understanding how the cabling in the connectome doesnt really tell us what the individual neurons do, just a wiring that send signals all across the connectome.
1. Why this matters
Flywire is trying to understand how the brain works and this could solve many neurological diseases. Today, the main approach is connecome scanning, but it struggles with speed. I chose to replicate the fly’s brain because it really interested me how I could literally download neurons and use them as a circuit.
2. What I replicated
The original work is “the fly project” by flyproject.io (https://flyproject.io). They showed that the fly can have different behaviors when faced with different scenarios and different objects that trigger motor functions. My goal was to reproduce the model and understand how the fly’s brain works.
3. Technology track
Tools and stack:
Minecraft
How it works:
The fly’s connectome provides a map of how its neurons are connected. The program uses this wiring information to simulate neural activity, allowing the fly to receive information from its virtual environment and produce behaviours inside Minecraft. This allows the fly to interact with the world and learn from its experiences.
Key components and what each does:
- FlyWire connectome: Provides the neural wiring diagram, showing how neurons connect through synapses.
- Neural simulation: Uses the connectome to simulate how signals travel through the fly’s neural network.
- Minecraft: Provides the virtual environment where the fly can move, explore and interact.
- Compatibility mods: Allow the neural simulation and Minecraft to communicate with each other.
- Learning system: Allows the fly to adapt its behaviour based on interactions with the environment.
How I tested it works:
Get Antonio Heras’s stories in your inbox
Join Medium for free to get updates from this writer.
I ran the simulation inside Minecraft and observed how the fly interacted with the virtual environment. I checked whether its neural network produced different behaviours depending on the information it received and whether it could adapt through experience.
Data or inputs it uses:
- FlyWire’s neural connectivity data.
- Neuron and synapse information from the fly’s connectome.
- Sensory information from the Minecraft environment.
- Environmental feedback, such as obstacles, movement and interactions.
- Experiences that influence its behaviour and learning.
4. Results
I finally got the fly in my minecraft world. Compared to the original, my result was a success aswell because the fly responded correctly to each test based on its primary functions and survival skills.
5. What went wrong and how I fixed it
The problem was that I didn’t know how to put the fly’s brain inside the simultation, but what really helped me was a mod created by: theflyproject.
This basically joins the connectome with the game, making the fly actually move inside of minecraft. It also tells it what to do with each neuron.
6. Limitations
My version is limited because I don’t know the full wiring that tells the fly where to send each signal, or how the objects ingame interact with the wiring, or how the information is sent. I can’t yet claim ownership of the project as I would have to manually program each neuron’s function. To be more confident, I would need more time and hours and hours of more research as it is an incredibly hard project.
7. What I learned
The most surprising thing was that I could actually achieve my goal, and as I really like this game, it suprised me that it could intervene in my studies some time. If I did it again, I would put more research into how theflyproject was made.
8. Next steps
For my next step, I want to create a small neural network, which would test how the connectome works and would interfere with the signals recieved and sent from the virtual entity.
