Apple announcing ARKit earlier this year at WWDC might have been the company’s first of steps in the Augmented Reality space but releasing it to developers certainly opened it up to endless possibilities. As they say, not all minds function alike, and developers with out-of-the-box ideas continue to raise the bar on what the platform can really achieve.
According to a 9to5Mac report, developer Osama Abdel-Karim was able to draw a 3D painting using ARKit. What’s more interesting is the fact that Karim drew the 3D painting without using any companion equipment and just with his fingers on a physical notebook. As seen in the latest ARKit demo footage, the user simply drew lines and figures in air using his fingers in the same way we use a pen. The program follows the finger movement and ink appears, looking quite realistic. Also, the user was able to add effects to bring the AR-based 2D painting to a visually appealing 3D object.
Watch the video here:
The 3D finger painting demo utilizes a combination of ARKit programs and the Apple’s Vision tool introduced in iOS 11. The combined algorithm tracks the fingertip resulting into realistic finger painting effect. The developer was kind enough to share a breakup of the whole process for us to understand. ALSO READ: A Tesla fan couldn’t wait for the Model 3 so he used Apple ARKit to bring one to his driveway
“One of the cool libraries that Apple introduced in iOS 11 is Vision Framework. It provides some computer vision techniques in a pretty handy and efficient way. In particular, we are going to use the object tracking technique. Object tracking works as follows: First, we provide it with an image and coordinates of a square within the image boundaries for the object we want to track. After that we call some function to initialize tracking. Finally, we feed in a new image in which the position of that object changed and the analysis result of the previous operation. Given that, it will return for us the object’s new location.”