Combining Image Classification with self driving

[Avinash279939]

I need to know , can it is possible to do image classification task to detect objects separately and parallel reinforcement learning algorithm for self driving. IF so where should i code. Do i need to modify in computer vision module for image classification and where to code RL task. Is it can be code in hellodrone.py both image classification task and RL task.

[Roopesh-Bharatwaj-K-R]

Hi @Avinash279939 , i too have this doubt , but he is the right person @ahmed-elsaharti to tell about this .

[ahmed-elsaharti]

Hi @Avinash279939 and @Roopesh-Bharatwaj-K-R, you don't necessarily need to modify the example scripts to make something work with AirSim. You could write your own script/program and all you'll need to get AirSim connected is (in python for example:)

import airsim
client = airsim.MultirotorClient()

Since a single AirSim instance allows for multiple connections (as far as I remember), you could always just have two separate python scripts connect to the AirSim simulation and perform different tasks where one is doing RL driving and the other grabs images through the API for classification. Both of these could be done in multirotor mode and/or car mode rather than computer vision mode.
Alternatively, you could maybe do both tasks from a single python code with multiple threads for multiple tasks, etc.

[Roopesh-Bharatwaj-K-R]

Hi, @ahmed-elsaharti , Thank You very much for the brief information of the process and really helpful.

[Avinash279939]

@ahmed-elsaharti , Thanks a lot for easy explanation of the process , And i want to ask you one, is there any examples of RL task built by Airsim for understanding purpose or any other examples/ Pre-trained models which are available opensource., Thanks Again.

[jonyMarino]

Hi and welcome @Avinash279939! Take a look at these examples:
https://github.com/microsoft/AirSim/tree/master/PythonClient/reinforcement_learning
We don't have Pre-trained models.

[Roopesh-Bharatwaj-K-R]

@ahmed-elsaharti , @jonyMarino , Hi , thanks for the information, I tried running the DQN_Drone.py
(https://github.com/microsoft/AirSim/blob/master/PythonClient/reinforcement_learning/dqn_drone.py)

it returned a cuda error (Could not load dynamic library Cuda64_101.dll) , but i have installed the cuda drivers and all packages necessary for it and i have built the Nbody.snn, then i run that, But still when i run on airsim... pops this error.
Any suggestions for this to resolve.

Computer Spec: RTX 2060 , 16 Gb Ram

Below is the error image.