Running the Simulator

This guide details the steps required to run the simulator.

1. Run the Simulator

If you're utilizing Azure OpenAI services for user_llm, ensure you disable the default jailbreak filter before running the simulator.

Use the following command to run the simulator:

python run.py --output_path <output_path> [--config_path <config_path>] [--dataset <dataset>]

Arguments: - --output_path: (Required) Path for saving output files - --config_path: (Optional) Path to config file (default: config/config_default.yml) - --dataset: (Optional) Dataset name (default: 'latest')

Example:

python run.py --output_path results/airline --config_path ./config/config_airline.yml

Troubleshooting
- Rate limit messages → Decrease num_workers variables in the config_default file.
- Frequent timeout errors → Increase the timeout values in the config_default file.

2. View Results

After simulation completion, results will be organized in the following structure:

experiments/
├── dataset__[timestamp]__exp_[n]/    # Experiment run folder
│   ├── experiment.log                # Detailed execution logs
│   ├── config.yaml                   # Configuration used
│   ├── prompt.txt                    # Prompt template
│   ├── memory.db                     # Dialog memory database
│   └── results.csv                   # Evaluation results

datasets/
├── dataset__[timestamp].pickle       # Dataset snapshot
└── dataset.log                       # Generation logs

policies_graph/
├── graph.log                         # Policy graph logs
└── descriptions_generator.pickle     # Generated descriptions

To visualize the simulation results using streamlit, run:

cd simulator/visualization 
streamlit run Simulator_Visualizer.py

This will launch a Streamlit dashboard showing detailed analytics and visualizations of your simulation results.