Dominik Kulon

Turn yourself and your friends into digital characters with pixel-level accuracy, in real time, right on your phone. With just a single image, creators can unlock immersive, full-body transformations.

• 3D Hand Tracking
• Body + Hands

I obtained a PhD degree under the supervision of Prof. Michael Bronstein and Prof. Stefanos Zafeiriou in the Department of Computing at Imperial College London.

• Thesis

We introduce a simple and effective network architecture for monocular 3D hand pose estimation consisting of an image encoder followed by a mesh convolutional decoder that is trained through a direct 3D hand mesh reconstruction loss. We train our network by gathering a large-scale dataset of hand action in YouTube videos and use it as a source of weak supervision. Our system largely outperforms state-of-the-art methods, even halving the errors on the in-the-wild benchmark.

• arXiv
• Dataset
• BibTeX

• AR/VR Workshop Video

Monocular 3D reconstruction of deformable objects, such as human body parts, has typically been approached by predicting parameters of heavyweight linear models. In this paper, we demonstrate an alternative solution that is based on the idea of encoding images into a latent non-linear representation of meshes.

• arXiv
• BibTeX

• Code
• Poster

The thesis presents a system that establishes a correspondence from hand pixels in an RGB image to the 3D hand model in an end-to-end manner. It is accompanied by a hand model capable of representing different shapes learned from hand scans and the first dataset of 1,500,000 hand models optimized to match the pose and shape of hands in color images.

• Presentation