Environments
The environment is what the agent interacts with, reading from it via its percepts, and writing to it via its actuators. For example, a grand-theft-auto character in the game while out on the streets, it would be the streets themselves, the cars, buses and planes, the weather, the people on the streets, gang members, drug-dealers, hookers, and the corrupt police.
Being able to understand the environment and classify it is imperative as it strongly influences the design of the agent.
Properties of Task Environments
- Fully Observable (Accessible) vs Partially Observable (Inaccessible)
- Deterministic vs Stochastic (Non-Deterministic)
- Episodic vs Sequential (Non-Episodic)
- Static vs Dynamic
- Discrete vs Continuous
- Single-Agent vs Multi-Agent
- Simulated vs Situated or Embodied
- Active vs * Passive
- Strategic Environments
As one might expect, the hardest environment would be a partially observable, stochastic, sequential, dynamic, continuous, multi-agent, and embodied one. As it turns out, this is exactly the "real world" in which we live, which is why simulations are much easier to implement, than real-world robots.
Example Combinations
- Static and Continuous - Snooker - The player can be assured that the environment is static (i.e. no one will come and pocket another ball while he is deliberating his next move), and continuous because, for example, the agent can hold the cue, and manuver it from various angles at various velocities, there's no discrete set of moves that he's restricted to.
Examples
| Chess | Poker | Robocup Soccer | Internet Shopping | Backgammon | |
| Fully Observable | |||||
| Deterministic | |||||
| Episodic | |||||
| Static | |||||
| Discrete | |||||
| Single-Agent | |||||
| Simulated |
