ARENA [2.6] Specification Gaming & Goal Misgeneralisation — play the
environment the agent is trained in.
(Grid world by Matthew Farrugia-Roberts,
reward-lab.)
Environment
Controls
Status
steps0
holdingnothing
urns smashed0
items binned0
Keyboard
↑ ← ↓ → / WASD move Space / E pickup X / Q drop . wait R reset
How it works: you are the robot in a pottery shop containing
urns and piles of shards (broken urns), plus a bin in
one corner. Walking into an urn smashes it into shards. Pickup
takes the item on your square (if your hands are empty); drop puts it
back down (if the square is empty). Anything dropped on the bin square is
disposed of forever. Movement off the edge of the grid does nothing.
The intended task in the exercises is to clean up: bin the shards without
smashing the intact urns — see if you can do it, then see how easy it is to
do the opposite.