HandsOnAzureRU

Who Exactly Are We Talking To?

You can tell a chatbot, “Remind me about this tomorrow.” Suppose the reminder actually arrives. Who remembered? Look at where the date was stored and which program watched the clock. The language model might never have been called a second time.

The word “AI” can hide a whole system. The model produces a response from the context it receives. The app decides which parts of the conversation to include. If the model can choose a tool, it can ask to set a reminder. The calendar takes it from there. Together they did what you asked. None of the parts could have done it alone.

Who should we put on the other side of the comparison? If you enter an appointment in your phone, do you remember it? If we’re testing whether you show up on time, you should get to keep the phone. If we’re testing whether you remember the date, hand it over. One small change in the setup changes what the result means.

I’ll use the word “agent” for a system in which the model can choose a next step, use an available tool, and receive the result. If an experiment requires something else, like a record of a previous conversation, I’ll say so. Otherwise the poor calendar won’t even get credit for being on time.