Following a CV-related inquiry, the computer vision team at Dataart created a custom solution for detecting, capturing, and tracking visual markers from a real time video stream. What is usually called Augmented Reality and typically associated with a 3rd party ‘black box’ which does all the complicated spatial job, is now modeled and implemented from scratch by DataArt, with all the 3D mathematics created by researches and ported and implemented by developers.
Having performed many experiments on various existing AR engines suitable for the iOS platform, the team found that there is no solution that perfectly fits the specific task of capturing variable markers. Unlike AR programs where the marker’s shape and inner details are fixed, the current R&D is targeted to work with markers that have their outer contour unchanged, yet may be filled with different inner content – i.e. a user’s picture in a frame of a special form, etc. This hinders the use of existing AR engines as they do not expose the means to control marker digitizing and for the capturing process to consider only constant parts for dynamic 3D object matching. What is also related to using variable markers is that one typically needs to know what is inside of the marker that has been captured. The research on the existing engines has shown that they are missing this feature, which is essential for the above case.
Following multiple experiments with 3rd party solutions, a fully custom AR solution prototype was created by the DataArt CV team which allows effective working with variable markers. The AR prototype is implemented for the iOS platform, with the extensive use of a domain-standard open-source framework, the openCV, as a set of ready-to-use algorithms for image processing, 3D transformations etc. The spatial mathematics the engine is built upon was first prototyped and tested in Matlab.
The current prototype implements target marker capturing, stabilization, and tracking, with the ability for the captured object to be converted to a 2D marker plane, reconstructed, and returned to the user code. The latter feature, missing in all other AR engines, is obligatory when dealing with variable markers, where the actual content of the seized object matters as well as the 3D position.
The prototype mathematics was state-of-the-art coded the way that it allows the capturing and tracking of a 3D marker without a reduction of the default FPS rate of 30 per second on modern iOS devices such as Apple iPhone 4, and Apple iPhone 5.