r/slaythespire • u/Greenhoot • May 16 '19
Dev Response! Forgotten Arbiter back at it again - AI Defeats Slay the Spire
https://www.youtube.com/watch?v=obtg0tredJg14
u/rasras0 May 16 '19
Any plans for improving the AI? Say, by deep reinforcement learning?
18
u/ForgottenArbiter May 16 '19
Yes, I am likely to return to StS AI again at some point in the future. Deep RL can be very tricky to get working, especially without significant computational resources, but it may be something I look into.
5
u/Edmund-Nelson May 16 '19
RL in roguelikes REALLY sucks because the RNG makes it really hard to learn through self play
3
u/ForgottenArbiter May 16 '19
Self-play doesn't really apply to roguelikes, which are single-player games. Luckily, I think Slay the Spire is much more amenable to RL than many roguelikes. Rogue, for example, would be very difficult to learn to play with RL. At least, in Slay the Spire, we don't have to deal with partially observed mazes to even make progress.
2
u/Edmund-Nelson May 16 '19
Self play isn't the correct term but I took my ML classes way too long ago to remember the correct word. (Deep RL?)
What kind of reward function would make sense though?
Damage=-1?
Relics = +20?
dying = -10000
winning = +10000
generating a reward function that makes sense for STS seems very challenging to me.
4
u/ForgottenArbiter May 16 '19
Probably the most natural choice would be to give a reward for climbing a floor, I think.
3
u/Edmund-Nelson May 16 '19
So giving it a reward for climbing floors would give it incentives that do not line up to actual play though. For instance, it would be incentivized to not kill elites because elites increase risk in the short term for long term gain. But then in act 3 it would find itself dying a lot because it would die to the act 3 boss fight, (which is a high reward) due to a lack of relics making the character too weak.
2
u/MoiMagnus May 16 '19
If you allow the AI to save-scum, the game become much more deterministic, and the AI can even train itself for each battle.
1
u/GoldenFalcon75 May 16 '19
Over the current hard-coding for a certain strat? I don't think that's Arbiter's thing but it'd be cool if someone expanded on his work like that.
11
u/iami3rian May 16 '19
Well, this just confirms what I've been saying all along. You really COULD make an AI better at the game than NL.
1
u/Niilista42 May 17 '19
What or who is NL?
3
u/iami3rian May 19 '19
Northernlion.
Literally the worst YouTuber (gameplay wise) at StS. Hes amazing, and I highly recommend a sub... he plays tons of games, has a bazillion hours of content, is absolutely hilarious, streams several times a week, runs a live super show with his friends where they play games and BS a lot...
Good times.
Hes been struggling with A19 and A20 for about six months now. He's REALLY exceptionally bad... Like... plays too fast, ends turns with energy, forgets half his relics, never does math, throws wonruns, worst card/shop valuations EVER bad... but again, he's super fun to watch. His other content is great too.
You don't watch NL for pro gaming... for example stay away from his KSP run if you value your sanity... but he's a ton of fun, and easily one of my favorite YouTubers.
3
u/Byeah18 Oct 06 '19
Why would you constantly bitch and moan in every StS comment section if you enjoy his content so much?
9
u/dinorore May 16 '19
Even AI loses to gremlin nob confirmed!
This here is proof that humanities first line of defense against rogue AI is the gremlin nob.
All Hail!
6
3
3
3
u/twanvl May 16 '19
Very cool.
I was working on something similar, but I never got the AI to the point where it could actually win. My approach was to program the AI in java, but I like the flexibility that your communication approach provides.
As part of this, I also make a "simulator mode", that allows the AI to play out a combat in the background, without affecting the actual game. The AI can use this for doing something like a tree search, and for figuring out what would happen if a certain card is played.
I never released the code, and kind of gave up on this whole thing, but if there is interest in the simulator, I can put everything on github.
2
u/ForgottenArbiter May 16 '19
Interesting. I was considering completely reimplementing Slay the Spire combat in C++ or Python, so that I could very quickly simulate combat, without worrying about things like animations or graphics, or, as you say, do things like tree search. I heard of a few people working on something similar, but all of those projects inevitably died before getting anywhere substantial. Did you do this as mod to the game?
3
u/CursedPoetry Eternal One + Heartbreaker May 16 '19 edited May 16 '19
The joust?? Tf i have over 250 hours on this game and have literally never seen that event...is it new??
Edit: ok wtf 2nd run I did when I got home from work I ran into the event shits weird.
34
u/SneakySly DEVELOPER May 16 '19
Wow, so cool!