Data-driven simulation for training and evaluating full-scale autonomous vehicles.

Related tags

Admin Panels vista
Overview

VISTA Driving Simulator

Docs PyPIDownloads Stars PyPI

https://raw.githubusercontent.com/vista-simulator/vista/main/docs/source/_static/overview.jpg

VISTA is a data-driven simulation engine for autonomous driving perception and control. The VISTA API provides an interface for transforming real-world datasets into virtual environments with dynamic agents, sensor suites, and task objectives. Because VISTA is data-driven, it is side-steps many of the traditional issues of simulators, such as their lack of photorealism and ability to accurately model reality.

Installation

VISTA can be installed into your Python 3 environment using the PyPi package interface.

>> pip install vista

Please also ensure that you have all required dependencies to successfully run VISTA. Details on dependencies are outlined in the documentation.

paper1 paper2 paper3

Citing VISTA

If VISTA is useful or relevant to your research, we ask that you recognize our contributions by citing the following three original VISTA papers in your research:

% VISTA 1.0: Sim-to-real RL
@article{amini2020learning,
   title={Learning Robust Control Policies for End-to-End Autonomous Driving from Data-Driven Simulation},
   author={Amini, Alexander and Gilitschenski, Igor and Phillips, Jacob and Moseyko, Julia and Banerjee, Rohan and Karaman, Sertac and Rus, Daniela},
   journal={IEEE Robotics and Automation Letters},
   year={2020},
   publisher={IEEE}
}

% VISTA 2.0: Multi-sensor simulation
@inproceedings{amini2022vista,
 title={VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles},
 author={Amini, Alexander and Wang, Tsun-Hsuan and Gilitschenski, Igor and Schwarting, Wilko and Liu, Zhijian and Han, Song and Karaman, Sertac and Rus, Daniela},
 booktitle={2022 International Conference on Robotics and Automation (ICRA)},
 year={2022},
 organization={IEEE}
}

% VISTA 2.0: Multi-agent simulation
@inproceedings{wang2022learning,
 title={Learning Interactive Driving Policies via Data-driven Simulation},
 author={Wang, Tsun-Hsuan and Amini, Alexander and Schwarting, Wilko and Gilitschenski, Igor and Karaman, Sertac and Rus, Daniela},
 booktitle={2022 International Conference on Robotics and Automation (ICRA)},
 year={2022},
 organization={IEEE}
}

Contribution Guidelines

VISTA is constantly being advanced and has been built with research, extensibility, and community development as a priority. We actively encourage contributions to the VISTA repository and codebase, including issues, enhancements, and pull requests.

You might also like...

Analysing the US Bike-share data and deriving deeper data-driven insights from the same using python

US-Bikeshare-case-study Analysing the US Bike-share data and deriving deeper data-driven insights from the same using python The case-study was taken

Aug 15, 2022

DeepGNN is a framework for training machine learning models on large scale graph data.

DeepGNN Overview DeepGNN is a framework for training machine learning models on large scale graph data. DeepGNN contains all the necessary features in

Jan 4, 2023

This repository presents a full process of training a pedestrian tracking model using `deep-person-reid` repository. Further, a complete web-application service based on FastAPI is provided that extracts features using YoloV5 and trained model.

This repository presents a full process of training a pedestrian tracking model using `deep-person-reid` repository. Further, a complete web-application service based on FastAPI is provided that extracts features using YoloV5 and trained model.

Person Tracking DeepSORT Yolov5 Person tracking/re-identification is one of the major deep learning problems that is very popular in surveillance appl

Jun 30, 2022

Python tools for running and evaluating DM-VIO.

Python tools for DM-VIO: Delayed Marginalization Visual-Inertial Odometry When using this project in academic work, please consider citing: @article{s

Nov 22, 2022

Evaluating Vision & Language Pretraining Models with Objects, Attributes and Relations.

Evaluating Vision & Language Pretraining Models with Objects, Attributes and Relations.

VL-CheckList Updates 07/04/2022: VL-CheckList paper on arxiv https://arxiv.org/abs/2207.00221 07/12/2022: Updated object, relation, attribute splits/d

Dec 29, 2022

Official implementation of SRCN3D: Sparse R-CNN 3D Surround-View Cameras 3D Object Detection and Tracking for Autonomous Driving

SRCN3D: Sparse R-CNN 3D Surround-View Cameras 3D Object Detection and Tracking for Autonomous Driving This repo is the official implementations of SRC

Jan 3, 2023

Implemented visual-inertial SLAM using an extended Kalman filter using IMU and stereo camera measurements from an autonomous car. First performed IMU localization via EKF prediction, then landmark mapping via EKF update.

Implemented visual-inertial SLAM using an extended Kalman filter using IMU and stereo camera measurements from an autonomous car. First performed IMU localization via EKF prediction, then landmark mapping via EKF update.

Visual-Inertial-Simultaneous-Localization-and-Mapping Implemented visual-inertial SLAM using an extended Kalman filter using IMU and stereo camera mea

Oct 16, 2022

MUAD: Multiple Uncertainties for Autonomous Driving, a benchmark for multiple uncertainty types and tasks [BMVC 2022]

MUAD: Multiple Uncertainties for Autonomous Driving Dataset This repository contains some development kits (in PyTorch) that we used for our BMVC 2022

Nov 25, 2022
Comments
  • How to source the data from the example files

    How to source the data from the example files

    Hi guys, congrats on making such an interesting project.

    How can we source the data from the example files? I got the vista_traces file but i dont know how to connect it to the program. When i import vista i get: 2022-09-04 19:37:06,783::WARNING::[vista.entities.sensors.EventCamera.<module>] Fail to import module for event camera. Remember to do source <some-dir>/openeb/build/utils/scripts/setup_env.shCan ignore this if not using it

    There must be a simple solution to this problem, i searched the docs but i could not find the explaination for this.

    opened by Philan014 0
  • How to use RGBD and Stereo cameras in this software?

    How to use RGBD and Stereo cameras in this software?

    Hello,

    Congratulations on a great effort and thanks for sharing it with the public. I have a query regarding the sensor suite. Query:

    1. As there are handful of proprietery cameras like Intel D4115, D435, ZED1 and 2, OAK-D etc. how should someone be able to use those camera models into the current software stack of vista2.0.5?

    Eager to know from you,

    opened by ahar 0
  • error running sim_lidar.py basic usage examples..

    error running sim_lidar.py basic usage examples..

    Hello, thanks for this work it's great !! For my project, I wanted to test the lidar sim functionality. I am running into some issues, seems like some resource files related to Lidar aren't pushed, which seems to be causing the issue. the error I am getting. self.avg_mask = np.load(str(rsrc_path / "Lidar/avg_mask2.npy")) File "/home/amohan/python_env/vista_env/lib/python3.8/site-packages/numpy/lib/npyio.py", line 390, in load fid = stack.enter_context(open(os_fspath(file), "rb")) FileNotFoundError: [Errno 2] No such file or directory: '/home/amohan/python_env/vista_env/lib/python3.8/site-packages/vista/resources/Lidar/avg_mask2.npy'

    As you can see from the LidarSynthetisis code it requires some files image kindly push or provide a way to download those files if they already exist point me towards them. Thanks in advance.

    opened by AdithyaVenkateshMohan 1
  • How to compute quaternion parameters if I know camera position and Euler's angles?

    How to compute quaternion parameters if I know camera position and Euler's angles?

    I've been trying to learn the control policy for custom datasets. And I can get quaternion values right in the file params.xml. I either get the wrong translation or even though visually the translations are more or less similar to the translations with the vista dataset. Nonetheless, the total reward eventually stuck around five and never recover afterward.

    opened by rabdumalikov 0
Releases(2.0.5)
Simulation of Autonomous search and rescue operation in a fire outbreak environment using Deep Reinforcement learning. (Project under development 💻)

Autonomous Search and Rescue Agent using Deep Reinforcement Learning Domain: Artificial Intelligence Problem Statement: We are living in a world full

SRI JAYAN E 3 Nov 30, 2022
A full python simulation of gravitational interaction between planets of different mass and size, colliding in an elastic way.

Planetarium A full python simulation of gravitational interaction between planets of different mass and size, colliding in an elastic way. Hi ! Here i

DaCapo 3 Dec 7, 2022
Calibration-free traffic state estimation method using detectors and connected vehicles data

FreeTSE Calibration-free traffic state estimation method using detectors and connected vehicles data Executable version with manual Download How to us

Toru Seo 1 Sep 29, 2022
Training and evaluating NBM and SPAM for interpretable machine learning.

Neural Basis Model (NBM) and Scalable Polynomial Additive Model (SPAM) Training and evaluating NBM and SPAM for interpretable machine learning. Librar

Meta Research 57 Dec 25, 2022
University project for training and evaluating reinforcement learning agents in OpenAI Gym environments using DQN.

Reinforcement Learning Gym Train and evaluate reinforcement learning agents in OpenAI Gym environments using Deep Q-learning (DQN). Environments The f

Adam Ross 2 Sep 5, 2022
Processing crowdsourced bathymetry from International Hydrographic Office's Data Centre for Digital Bathymetry (CSV data processing). Corrects for water levels/tides and data-driven/postulated vessel transducer offset values.

Crowdsourced_Bathy_Processing Processing crowdsourced bathymetry from International Hydrographic Office's Data Centre for Digital Bathymetry (CSV data

Anthony Klemm 2 Dec 11, 2022
Data Cleaning. Data Integration. Data Reduction for 1-Data Quality. 2-Data Transformation. 3-Data Mining. 4-Pattern Evaluation. 5-Representing Knowledge in Data Mining.

Python_application_for_dataMining Oreview: We aim to achieve a prediction model for improving data analysis and reporting. The programming language th

abdelghani 1 Sep 21, 2022
Detect pedestrians, cars, trucks, bicycles and other vehicles on the road where the vehicle is traveling, and issue an alarm

Road-condition-detection ?? ?? Detect pedestrians, cars, trucks, bicycles and other vehicles on the road where the vehicle is traveling, and issue an

Buqi 6 Aug 11, 2022
Tabular Traffic Flow Model Guided Path Planning with Connected vehicles.

Tabular Traffic Flow Model Guided Path Planning with Connected Vehicles The data of network (SUMOFiles/node.csv, SUMOFiles/link.csv, SUMOFiles/demand.

Darco 2 Aug 25, 2022
LPR (License Plate Recognition) is an image-processing technology used to identify vehicles by their license plates

LPR (License Plate Recognition) is an image-processing technology used to identify vehicles by their license plates. This technology is used in various security and traffic applications, such as the access-control system featured in the following animation:

Amrith 1 Nov 6, 2022