The Pottery Shop

ARENA [2.6] Specification Gaming & Goal Misgeneralisation — play the environment the agent is trained in.
(Grid world by Matthew Farrugia-Roberts, reward-lab.)

Environment

Controls

Status

steps0
holdingnothing
urns smashed0
items binned0

Keyboard

↑ ← ↓ → / WASD  move
Space / E  pickup
X / Q  drop
.  wait    R  reset
How it works: you are the robot in a pottery shop containing urns and piles of shards (broken urns), plus a bin in one corner. Walking into an urn smashes it into shards. Pickup takes the item on your square (if your hands are empty); drop puts it back down (if the square is empty). Anything dropped on the bin square is disposed of forever. Movement off the edge of the grid does nothing. The intended task in the exercises is to clean up: bin the shards without smashing the intact urns — see if you can do it, then see how easy it is to do the opposite.