Evaluating SYNGrid : A Configurable Benchmark Environment for TemporalCredit Assignment in Reinforcement Learning

Sammanfattning

A central challenge in Artificial Intelligence (AI) is assigningcredit to the correct actions within long chains of events thateventually lead to a reward, something Minsky noted backin 1961. This is the heart of the Credit Assignment Problem(CAP), and targeted benchmarks that isolate it fromexploration challenges are still lacking.The goal of this thesis is to evaluate SYNGrid, a configurablegrid-based AI benchmark framework, focusing on its tieredorb mechanic and configurable delay as controlled measuresof Temporal Credit Assignment (TCA).Three types of agents are compared to assess if SYNGridproduces meaningful differentiation across architectures.The agents include a stateless baseline, a frame-stackingagent, and a Long Short term Memory (LSTM)-based agentsharing the same underlying algorithm. TCA is assessed bycomparing agent performance under three distinct scenariosin the environment.The benchmark design is also evaluated for its ability toproduce distinct and meaningful task variations withoutcode changes.Results show that the stateless agent outperformed bothmemory-augmented agents across most scenarios, withLSTMs advantage emerging primarily under partialobservability where its ability to reconstruct spatial contextproved decisive, however, higher explained variance andlower entropy across all scenarios suggest a more structuredunderstanding of the task compared to the other two. Framestacking offered marginal benefit in the same task anddegraded performance elsewhere. These distinctions weremade possible through configuration alone, though asprawling config file places cognitive load on the user that adedicated interface could reduce.SYNGrid produces measurable differentiation across TCAchallenges, partially isolating it from exploration andperception across three evaluated scenarios. Entanglement ispossible, as the spatial scenario illustrates, but reflects theframework’s expressive range rather than a fundamentallimitation. Scenario construction through configuration aloneproved functional and flexible, at the cost of cognitiveoverhead that a dedicated interface could address.