Skip to content

Treating noise and anomalies in the vehicle time-series data captured by drones

License

Notifications You must be signed in to change notification settings

vishalmhjn/pneuma_treatment

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

48 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

pneuma_treatment

Copyright 2020-2023 Vishal Mahajan

Trajectory data collected from Drones over an urban area in Athens. For more info: https://open-traffic.epfl.ch/

drawing

This repository provides an automated way to treat the noise and anomalies (in the form of unrealistic peaks) in the acceleration values of a number of vehicles in the pneuma dataset. The algorithm uses a combination of low-pass filters (Savitzky-Golay filter, Gaussian filter) to remove the noise. A machine learning model (XGBoost) is used to reconstruct acceleration time-series and detect anomalies. If anomalies are detected, they are removed. The outputs are the processed time-series profiles of vehicle speeds and accelerations without high frequency noise and unrealistic acceleration peaks.

Code

You should be in src folder when running the codes.

Raw data to Long format

  1. Specify the path to the raw data from pNEUMA website in the data_formatter.py and run it to convert the files from wide to long format. A sample output is provided.

Treating noise and anomalies

  1. Specify the path to the output file from step-1, other paths (to save output data and plots) in the processor.py. Run this file to get the treated speed and acceleration time series.

Outputs

With the default paths, the plots are saved in plots and data is saved in data/derived.

Processed data

A link to the processed speed and from the 1000-1030 hour recording is provided at an external link. This file contains only four columns: vehicle id, frame number, processed speed and processed acceleration. Using vehicle id and frame columns, this data can be merged with the original data.

Citation

If you use these codes in your work, kindly cite the following:

V. Mahajan, E. Barmpounakis, M. R. Alam, N. Geroliminis and C. Antoniou, "Treating Noise and Anomalies in Vehicle Trajectories From an Experiment With a Swarm of Drones," in IEEE Transactions on Intelligent Transportation Systems, doi: 10.1109/TITS.2023.326871.

Link to the published article.

Link to the accepted version for personal use.

About

Treating noise and anomalies in the vehicle time-series data captured by drones

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published