ISAT User's Guide

Introduction

What This Document Contains

This is the user's guide for the National Advanced Driving Simulator (NADS) Interactive Scenario Authoring Tool (ISAT), which allows graphical authoring of driving simulator scenarios as well as rehearsal and review of simulator runs. ISAT has been designed to use graphical menus, visual feedback, and guided input dialog boxes to increase usability. This guide describes how to use the ISAT and defines the authoring capabilities of the NADS Scenario Definition and Control (SDC) software.

Audience

This guide was written for an audience that‟s very familiar and comfortable with the use of a Windows based personal computer. It is expected, that readers of this document are familiar with general transportation concepts, high-fidelity driving simulation, immersive virtual environments, and the issues involved in designing scenarios.

Framework

The ISAT is one of a number of tools that can be used to prepare for conducting research studies at the NADS. Therefore, it is important to understand how the ISAT fits within the overall framework. Although it is difficult to pinpoint a process that is simple and easy to comprehend yet covers all potential models of NADS usage, we believe the example provided here covers a significant subset of NADS usage. This example should not be considered a limit on how the NADS can be used to investigate problems. Rather, it is an example that is useful in explaining the role of the ISAT (i.e., the process does not indicate the potential to instrument new cabs or integrate new in-vehicle devices to current NADS cabs, yet these activities are possible). At the same time, not all parts of this process are necessary for each NADS study, and some could be skipped under certain conditions. With those caveats in mind, Figure 1-1 illustrates this process.

The Tile Mosaic Tool (TMT) is a graphical tool that allows the creation of the road network that will be used in the simulator scenarios. The pool of tiles refers to a set of pre-fabricated components, each representing a small geographical area (e.g., a city block or a piece of road) that can be combined by the TMT into a larger virtual environment to be used in the NADS. The output of the TMT is referred to as a static virtual environment because it contains only the physical environment (i.e., the road network, buildings, terrain, and features) but no active elements (i.e., traffic, pedestrians, etc.). The static virtual environment comprises several distinct but correlated databases, each of which is used by the various NADS subsystems while the NADS is running. For example, the visual representation of the virtual environment is displayed by the NADS Image Generator and projectors, whereas the road network information is used by the various driver models populating the virtual environment. The ISAT depends on the existence of a static virtual environment on which to build scenarios. That file is called the LRI (Logical Road Information) file. LRI files typically use the .lri extension. The pool of scenario components represents existing libraries of scenarios that can be used for building new scenarios. The ISAT produces a scenario file, which, in combination with the static virtual environment, represents a dynamic virtual environment. This information is fed to the experiment builder, a tool that allows the specification of the number of participants in a study and the conditions that each participant will be exposed. For example, let us assume that the ISAT produces two scenarios, S1 and S2. If an upcoming study will use eight subjects, then each of the subjects can have any number of exposures using either S1 or S2. The experiment builder allows the specification of such associations. Upon completion of these associations, the cumulative set of files and databases is transferred to the NADS computer systems, where they can be used to execute the runs. After completion, the final output of the simulator is a number of raw data files, which may include video, binary, or other data. These data are then fed to various post-processing tools (this process is referred to as data reduction), and the resultant reduced data can be used by researchers for statistical analysis. Generally, the ultimate goal of this process is the generation of reports, articles, publications, or other publicity materials. This is facilitated through a Data Verification and Visualization Workstation (DVVW), which is a hardware-software custom-developed product that facilitates the integration, understanding, and dissemination of all available information.

Note that no matter how complex the data that is produced in the various phases of the process, the goal of the data-handling tools is to hide this complexity from the user.

Terminology

This section defines terms that are used frequently in this document.

Visual database	A set of data files representing the physical appearance of a virtual environment. The NADS computers use the OpenFlight format for visual databases.
Synthetic environment	A set of correlated databases that fully defines a virtual environment for use in the NADS.
Tiles	An instance of a synthetic environment that has been designed so that it can be combined with others, thus allowing the development of larger synthetic environments.
Simulator driver	The person driving the NADS. Depending on the context, the term may refer to the actual individual who is driving, or the vehicle representing the driver in the virtual environment. An example of the latter occurs when generating a top-down movie showing what happened during execution. The term simulator driver would be used to refer to the vehicle that shows the subject's actions.
Event	A pre-specified set of actions that may or may not involve the simulator driver and are an integral part of a scenario. It is often necessary to have repeatable events. An example of an event is having a car that is waiting at a red light run the light once the simulator driver is ready to enter the intersection.
Scenario	A series of events that take place while a subject is driving the simulator.
Scenario element	An object that usually is displayed in the ISAT main window, and corresponds to an entity that will exist in the virtual environment during execution of the scenario.
Research participant	An individual who drives the NADS during an experiment. Note that a single research participant may drive the NADS several times within the same experiment, and each time they may be exposed to a different scenario. Also called subject.
Subject	An individual who drives the NADS during an experiment. Note that a single subject may drive the NADS several times within the same experiment, and each time they may be exposed to a different scenario. Also called research participant.
NADS	The National Advanced Driving Simulator.
SDM	The Simulator Development Module, a second simulator in the NADS facility that allows testing and tuning of scenarios or other simulator modifications without occupying the NADS. A key feature of the SDM is that it uses hardware identical to or similar to that used in the NADS to ensure compatibility.
Coordinator	A coordinator is a scenario component that has no visual representation but can monitor the evolution of the virtual environment and create, delete, or control other entities to orchestrate scenarios.
Buttons and dials	In the ISAT, most objects have buttons and dials. When pressed or set, buttons and dials can modify the behavior of the object. Scenario events are often created by coordinators pressing the buttons and dials of objects at key points during the scenario.
GUI	A Graphical User Interface is term used to describe a user interface that relies on graphical symbols such buttons, text boxes, and drop down menus, and uses mouse or other pointer device (touch screen, trackball, and joystick). GUI elements refer to individual elements such as buttons.

In Case of Problems

At the time of this writing, most NADS tools have been tested to meet functional specifications and used extensively in research and studies conducted at the NAD. In the event that you experience any problems or would like to provide feedback, please contact Omar Ahmad at mailto:oahmad@nads-sc.uiowa.edu, or David Heitbrink at mailto:dheitbri@nads-sc.uiowa.edu. When reporting a problem, please provide the following information:

1. Your name and affiliation
2. The version of the ISAT that you are using (version information is available from the Help/About menu)
3. The scenario and LRI that causes the reported problem
4. A description of the problem
5. A series of steps that need to be performed to recreate the problem

All problems will be addressed in a timely manner in the form of a fix or a workaround.

Document Outline

The remainder of the document is organized into chapters, each covering a particular aspect of ISAT usage. Chapter 2 provides an overview of the issues involved in building scenarios for the NADS and a broad outline of the tools‟ capabilities. Chapter 3 provides an overview of the general graphical user interface features of the ISAT, including how to create new scenarios, save scenarios, navigate the synthetic environment, and so on. Chapters 4 through 9 focus on authoring components, and Chapter 10 is devoted to more advanced subjects.

Scenario Authoring Overview

Basics

Scenario authoring involves the specification of both the static and dynamic elements of a virtual environment so that certain events appear to anyone who drives the simulator under the specific scenario, regardless of small differences in driving style or timing. For example, consider the testing of a new in-vehicle warning device that provides various alarms to warn drivers of potential collisions. The researchers are interested in seeing how different warning signals perform as far as their ability to alert the driver and prevent a collision. To test such a device, one should be able to create potential collision situations. One way to do this is to have the subject drive along a road and through an intersection controlled by traffic lights while the light is green. As the driver nears the intersection, another car (car B), which is initially stopped by the red light on the other leg of the intersection, begins moving, thus blocking the subject‟s path. Let us consider the issues in building such a scenario.

The problem

By far the most critical aspect of such a scenario is timing the motion of car B so that it provides a consistent time-to-collision condition for all subjects, something that is necessary in evaluating the collision avoidance device. However, additional issues have to be considered. When was car B created? Is car B is the same make and model for all subjects, or does it change? How does one ensure that the traffic light is green when the subject approaches as opposed to yellow or red? What if different subjects travel at different speeds toward the intersection?

One approach to building such a scenario is to script everything explicitly that is dynamic about the virtual environment so that its timing is tied to the motion of the simulator driver. For example, time 0 is when the scenario starts, time 40 is when the driver reaches some fixed distance from the intersection, time 60 is when the driver's front bumper enters the intersection, and time 100 is when the driver reaches some point beyond the intersection. Scripting software can determine the value of time based on how fast or how slow the drivers moves and then trigger every change in the virtual environment based on the parametric time scale. For example, the light will turn green at time 20, and car B will begin moving at time 40 and reach the middle of the intersection at time 60. That way, everything is scripted, and no matter how fast or slow the driver moves, the event will happen consistently.

Such techniques are often used in simulators that use relatively short scenarios or involve few dynamic traffic elements. The technique is simple, easy to understand, and provides deterministic results. However, when considering such a technique for use in high-fidelity simulators, several problems arise. In high-fidelity simulators, scenarios are longer and generally involve multiple entities. Coordinating one or two elements through explicit scripting for a 5-minute scenario is reasonable, but coordinating 500 vehicles for a 45-minute scenario is daunting and virtually impossible. Note that the 500 vehicles may not be active at the same time; in fact, they may represent traffic in the opposing lane that is not an integral part of the scenario. Nevertheless, if scripting were the only tool available, it would have to be used for all vehicles. Another complication has to do with what happens after the event takes place. One may want to trigger different scenarios after the near-collision event, depending on the outcome of the first event. For example, if there was a near collision, a less severe event may need to take place at the next intersection, but if the subject veered away early enough, a more severe event may be necessary. Such decision-making cannot be programmed when scripting is the only tool.

An alternative approach is to use intelligent agents that populate the simulator‟s virtual environment and behave autonomously. For example, autonomous driver models can be used to control the various vehicles in the scene. In addition, an intelligent manager can control the traffic lights so their timing follows a specific pattern. Such an approach makes it easy to create scenarios involving multiple entities because the labor-intensive specification of the individual behavior of every entity is eliminated. In addition, the length of the scenario does not overly complicate the development of the scenario because autonomous entities can be created automatically throughout the scenario. However, various other complexities surface. For example, timing and coordination become harder to achieve. Consider the example described earlier. If a traffic light manager controls the state of the traffic lights based on a fixed pattern, there is no guarantee that the light will be green when the simulator driver approaches the intersection. Furthermore, if vehicles are autonomous, there is no guarantee that car B will be waiting at a red light because it may have decided to pick a different route earlier. Even if a car is waiting at the red light, subjects will travel at different speeds. Therefore, they will reach the intersection at different times, and the proper time for car B to block the intersection is not known a priori.

ISAT Toolset Approach

The approach used for scenario authoring in the NADS software and supported by the ISAT is a hybrid between the two techniques, with emphasis on using intelligent agents in conjunction with coordinators to ensure consistency. Specifically, the ISAT allows the user to create any number of entities that behave largely autonomously. In addition, the user can create coordinators; coordinators are "invisible" entities that exist in the virtual environment, but are not displayed. These coordinators orchestrate events by monitoring what happens and modifying the autonomous behavior of the remaining agents to achieve a pre-specified goal. Additional coordinators can be used to automate the generation of traffic, control the traffic lights, modify the environment conditions, and control numerous aspects of the simulator‟s operation. In addition, the ISAT allows the creation of purely scripted entities. The evolution of these entities is completely deterministic, and their timing can be either independent or dependent on the simulator driver's actions.

Deterministic objects, or Deterministic Dynamic Objects (DDOs), as they are referred to in the ISAT, are objects whose behavior is pre-scripted by the user. Specifically, the user can select a path and specify the velocity of the DDO at each point in the path. While the scenario is running, a DDO simply follows its path according to the user's specification. Dependent DDOs, or DDDOs, are similar in that they follow a specific path, but their velocity adjusts so they reach a specific point of their path at the same time another entity reaches another target point. DDDOs allow vehicles that follow a scripted path to behave consistently as far as their relative position to another object (including the simulator driver), independent of the variation in the other object's speed. Both DDOs and DDDOs lack collision-avoidance capabilities.

Autonomous Dynamic Objects (ADOs) use a sophisticated autonomous driver model that controls their motion. One can think of an ADO as a human driver that is driving around the virtual environment trying to reach its destination. An ADO will follow the rules of the road, including following the speed limit and respecting traffic lights, and will exhibit most behaviors exhibited by real drivers. A unique feature of the driver model used in the NADS SDC, however, is the ability to have its default behavior modified at runtime. For example, an ADO can be told to follow a specific speed independent of the external conditions, to change lanes, to take a particular turn, and so on. Such commands can be initiated by the researcher while the scenario is running, or can be issued by the various coordinators that can be authored a priori. The latter is by far the most common due to its repeatability. The key coordinators include the trigger, the traffic manager, the traffic source, and the traffic light manager.

The trigger is an entity that is given a series of conditions and a series of actions. It continuously evaluates its conditions and, when they are all true, performs the actions. Examples of conditions include another object reaching a specific point in the road network or a traffic light turning green. Various actions are available, including creating a new entity, deleting an existing entity, modifying the cycle of the traffic lights, terminating the simulation, causing system failures on the vehicle driven by the simulator driver, and issuing behavior modification commands to a set of autonomous entities. A detailed description of how the trigger operates and how the ISAT can be used to specify the trigger‟s events and actions is given later in this document.

The traffic manager is tasked with generating autonomous traffic to populate the area around the driver. Because of a finite limit on the computational resources of the simulator, it is not possible to simulate an arbitrary number of vehicles in real-time. Therefore, traffic is created only in the area around the driver. The traffic manager is given a specific traffic profile and a desired density, and it will create or delete objects as necessary to maintain that density.

A traffic source is another coordinator whose job is to create traffic. Unlike the traffic manager, however, it creates vehicles at a specific point in the database.

The traffic light manager is responsible for controlling the state of traffic lights in the scene. Through the ISAT, the user can program the traffic light manager to achieve just about any timing cycle. In addition, the traffic light manager, in conjunction with the triggers, allows the linking of traffic lights to achieve coordinated traffic light groups.