HandsOnAzureRU

How Do You Design an Experiment You'd Believe?

There’s an annoying problem with the Alder warehouse. If the agent ships the cup correctly, it’s easy to put “rule learned” in the log. But what if it would have picked Alder before we ever corrected it? What if we accidentally left the word “packing” from our explanation in the question?

We need orders the agent has never seen. We can swap the warehouse names. Write down the conditions before running anything, while we still don’t know which result we’ll find ourselves rooting for. Then look at the log. Did the agent retrieve the correction, or solve the problem some other way? Keep the right answers that came from an unexpected route, too. We might have been testing something quite different from what we intended.

In Sutton and Barto’s textbook Reinforcement Learning: An Introduction, an agent interacts with an environment and the consequences of its actions are evaluated through reward. That’s a particular learning setup. At the warehouse, all we’ve done so far is record a verbal correction and see whether it comes in handy later. One successful shipment tells us little about how the system learned, even if its customer is happy.

An action log won’t help much with the joke. We’d need readers who weren’t told whose line they were judging. In the initiative experiment the log is especially valuable: we can see who started the agent and who proposed the topic. Beside each experiment I’d write one short thing: what result would change my mind? If that box is blank, the run can wait.