This repository contains source files developed for my MSc dissertation titled, "Procedural Generation of Datasets for Training Hand Pose Estimation Systems", at the University of Dublin, Trinity College.
Table of Contents
Generated image example
The rise of computer vision systems has reshaped many industries, driven by the powerful capabilities of deep neural networks. However, as the complexity of these systems grows, so does the demand for larger datasets. However, the manual annotation of large-scale datasets comprising the diversity required by these systems is labour-intensive and time-consuming. This dissertation delves into the exploration of procedural generation techniques for hand pose datasets to overcome this challenge while also investigating the impact of controlled variations in detection quality and reliability, encompassing joint angles, wrist orientations, texture, lighting, and background variations, aiming to make it capable of handling diverse real-world settings. To assess the efficacy of the generated datasets, a state-of-the-art computer vision system is trained to detect key points in hand images using both the procedurally generated dataset and traditionally annotated datasets. Comparative analyses evaluate the trained system's performance on real-world data, comprehending the influence of procedural variations on its accuracy, robustness, and generalisation capabilities. In conclusion, this dissertation contributes to hand pose estimation by integrating innovative approaches for procedurally generating datasets. The findings underscore the importance of automated variations in dataset generation and offer insights into their impact on the quality of trained computer fabrication systems.
Abstract from [1]
- Programming Language:
- C++ (C++20 dialect)
- Graphics Library:
- OpenGL 4.1
- Compilation Environment:
- Xcode Version 14.3.1 (14E300c) on macOS Ventura 13.4.1, compiled for an Apple M1 Pro CPU/GPU
- Additional Libraries:
- Glew (version 2.2.0)
- GLFW (version 3.3.8)
- Programming Language:
- Python (version 3.9.6)
- Machine Learning Library:
- PyTorch (version 2.0.1)
- Clone this repository.
git clone https://github.com/borjagq/HandyVariations/
- The 3D model used in this software is not included in the repository due to commercial licensing. You can download the model from this link. Place the downloaded files in the models folder with the following names:
hand.fbx
textures/hand_specular.jpg
textures/hand_base_color.jpg
textures/hand_normals.jpg
-
Download a 'random' image dataset, such as unlabeled COCO images. Place the images in a dedicated directory with no other files.
-
Run the script
rename_rdm_bg.shlocated in theScriptsdirectory, followed by the directory where you stored the random images. Ensure no other files are present, as they will be renamed.
zsh rename_rdm_bg.sh /Path/To/Random/Images/
-
Open the Xcode project
HandyVariations.xcodeproj. -
Build it and run it.
- Download all the required Python modules.
pip3 install numpy matplotlib tqdm torch torchvision Pillow opencv-python
- Open the Jupyter Notebooks (.ipynb) and run them for dataset validation.
The software available in this repository has been specifically developed for the experiments conducted in the Dissertation [1] presented by the author to the University of Dublin, Trinity College in partial fulfilment of the requirements for the degree of MSc in Computer Science (Augmented and Virtual Reality). It serves as a dedicated tool tailored to address the requirements and methodologies outlined in the research.
Borja García Quiroga, MSc
- Github: @borjagq
- Website: borjagq.com
HandyVariations is licensed under the GNU GPLv3, see LICENSE for more information.