Entries in iPad4 (1)

iPad with a 3D display and UI: Why it would be a game changer


Reality is not flat. Photographic and artistic representations of the world around us are currently presented in just 2 dimensions because the technology is not yet mature enough to offer a compelling way to create and present these representations in 3 dimensions. This 2 dimensional representation of reality on screen and in print is a poor show, it's a problem that needs solving and we think that Apple will be the company to do exactly that. 

Before we go any further we feel compelled to address the common complaints of 3D, there are those that feel that its recent resurgance in our cinemas is something truly unholy. The simple fact of the matter is this: The human brain is used to getting different images through each eye, getting the same image through both eyes is unnatural. Yes, 3D is more natural than 2D.

Whilst we have all grown comfortable with 2D images, in print and on our displays, this is not natural, not at least for images that are supposed to be representations of real places and objects. It's no more natural than black and white television was back in the 1950's. Yes, artists may choose to create photos, paintings, illustrations and movies in 2D, but this should be a stylistic choice just as the use of black and white instead of full colour is.

We wish critics of 3D would offer more clarity of argument when denouncing 3D cinema and television. Yes, they may well have issues with the current technology or the way that it is marketed, but to declare 3D in general as a gimmick is just ridiculous.

Clues as to how Apple might solve this problem

Because of a large number of patent applications that Apple has submitted on the subject, we already know that Apple is working on several technologies that it could apply to the field of 3D displays and user interfaces. What combination of these technologies Apple might use is open to debate, but we can have a good guess at some that might make an appearance at some point within the next 3 to 5 years.

Autostereoscopy: With any autostereoscopic display, the greater the number of pixels packed into the display, the better. A common example of a lenticular autosterescopic display is the display found in the top half of the Nintendo 3DS. Though the native resolution of the 3DS display is 800 x 240 pixels, the effective resolution is just 400 x 240 pixels per eye. Hence, endowing the iPad with a Retina class display (2048 x 1536 pixels) is a massive step in the right direction. If Apple go down the route of a lenticular autostereoscopic display, like the Nintendo 3DS, then the effective resolution would be 1024 x 1536 per eye, that's still considerably higher than the current 1024 x 768 resolution of the current iPad display.

Eye Tracking: A number of Apple's 3D related patents mention face tracking technology. Face tracking, or essentially eye tracking, is an important part of presenting the illusion of looking into a real physical world using just a flat display. See the Parallax entry on Wikipedia for much more information on the subject.

3D Motion and Gesture Sensing: Interacting with the 3D environment is where things will start to get really interesting. Apple's multi-touch technology, no matter how sophisticated it is, is all about manipulating a virtual 2D environment. Apple will need to jump to an entirely new level of user interface if the iPad is to be a truly effective 3D content creation device. Something like Microsoft's Kinect technology is one solution, and fortunately Apple have been working on something very similar to Kinect for a number of years. While it might seem far-fetched to believe that Apple could bring full 3D motion and gesture sensing to the iPad anytime soon, one only has to look at how effectively the Kinect performs to see that it's not beyond that realms of possibility.

Processing Power: It's likely that the iPad 2 will still be the reigning king of graphical horsepower right up until the day before the iPad 3 is launched. Despite Android tablet manufacturers packing their tablets with ever more powerful GPU/CPU chipsets, the mighty A5 is still in a league of its own in this regard. Having the ability to push pixels around the screen at an alarming rate is a prerequisite for any device that has to generate 2 separate world views, one for each eye. Fortunately, as we have already noted Apple is on a good GPU/CPU development path.

What will a 3D iPad mean to artists

Having an iPad with a proper 3D display, however Apple decide to implement it, will be a massive boon to those looking to watch 3D movies and play 3D games, but how will it benefit artists?

We can think of many ways in which an iPad with a proper 3D display and 3D motion and gesture sensing UI could literally transform the act of content creation, but we'll highlight just one area as an example for now.

3D Painting: Imagine being able to create a 3D watercolour by gesturing in the space above the iPad display. You might start off with an overall light coloured wash in the top half of the scene to represent the sky. To do this you would paint with your 4 fingers at a distance of about 8" above the iPad display. The further away from the display your fingers are the deeper into the 3D scene your brush paints - effectively, it would be like painting into a 3D mirror image. As you completed your scene with you would paint in the details closer and closer to the viewer, all the time moving closer to the surface of the display. In short, this would be a brand new way of creating art - a cross between sculpting and painting. Good artists would still need a proper understanding of light and shade and certainly a full working knowledge of perspective, but think of some of the glorious artwork they could produce with a full 3D palette.

How long will we have to wait?

Apple's patent applications would suggest that it is aggressively pursuing 3D display technology. As we have already mentioned in our October 2010 article, "4 innovations that are guaranteed to be in your next iPad, and 2 possibles", we don't see it arriving before 2015. Apple may well be one of the last manufacturers to include such a display, but one thing is certain, when they do it will be revolutionary.