Getting the feel of virtual reality

A giant leap forward in the realism of virtual reality (VR) may be just around the corner as a team of European researchers near the completion of a pioneering project to add textures, lighting effects and ‘feel’ to computer-generated 3D models.

Launched in 2002, the RealReflect project was the first attempt to use a new image acquisition technique known as Bidirectional Texture Function (BTF) that captures the look and feel of different materials. When this IST programme funded-project ends this October it is expected to result in the first comprehensive application using BTF for industrial modelling.

The project partners have geared their work toward the automobile sector, where the system could revolutionise the development of new models of vehicles by dramatically cutting costs and time. It also promises to open new possibilities in architecture, and further down the line, in computer games and other graphics applications.

Adding realism to virtual reality

“RealReflect is a major advancement over traditional virtual reality modelling, which basically relies on simplifications of reality by describing optical properties of a surface by a 2D matrix of data that does not show the real effects of lighting,” explains project coordinator Attila Neumann at the Technical University of Vienna. “Traditional virtual reality modelling, despite its name, lacks the feeling of reality and is a poor representation of it because the way things look highly depends on how they are illuminated and from what direction they are being viewed.”

By taking those two aspects – lighting and viewing direction – into account, the RealReflect system is capable of acquiring and rendering in VR even the most subtle textures, from leather on a car seat and wood panelling on a dashboard to metallic paint or chrome on door handles. Textures can be acquired from physical samples and then rendered onto the 3D models.

“It is a much more powerful and demanding system than traditional virtual reality modelling, making it look real instead of simply believable,” Neumann says.

That in turn brings with it additional complications. In order to be able to realistically represent textures the system requires a thousand times more data than other VR modelling tools, leading the project partners to develop compression techniques for the BTF information. The compression allows the models to be viewed and worked on in real time.

“It would be pointless having all this data if it filled up your hard drive and proved impossible to manipulate,” the coordinator notes.

The project also developed methods to take a small acquired sample of a material and multiply it seamlessly on a 3D model, which when viewed would show not only the texture but also its appearance under different types of illumination from different angles.

Allowing immersive reality

The overall result is a 3D modelling tool that permits immersive reality, especially when visualised in a CAVE, a cube-shaped VR simulator that users can walk inside and see everything in three dimensions.

“I could go into a CAVE and sit in a car seat and see the car around me, it would be like being inside the vehicle. I could look at the finish of the dashboard, the position of the gear stick, the material used on the seats,” Neumann explains.

To date the ability to view a vehicle down to the finest detail has only been possible by physically building a prototype, a long and costly process.

“When a car company wants to make a new model around 50 prototypes of different designs are built, of those most will be rejected before the company reaches the final stage of choosing a model from maybe five examples,” the coordinator says. “With RealReflect there would be no need to produce 50 physical prototypes as they could be created and viewed virtually, requiring maybe only five or 10 real prototypes or even less to be produced.”

That translates into “enormous” cost savings for car manufacturers and reduces the time it takes to bring a new model to market. “To date 3D models have only been used from an engineering perspective, never to actually verify what the vehicle looks like – with RealReflect that can be achieved accurately,” Neumann says.

Besides displaying in detail the look of the vehicle, the system could also enhance safety by allowing designers to see the way different types of illumination reflect off its surfaces. This could, for example, allow designers to reduce potentially dangerous reflections on the windshield that may otherwise go unnoticed.

Beyond the automotive sector, the RealReflect system could also be applied to architecture, allowing architects to better visualise the appearance of materials used in construction, while offering clients the opportunity to virtually tour a building.

“In the future it could also be used in computer games and other graphics applications,” Neumann notes.

The project partners are currently drawing up a commercial strategy to market the system, which could include either selling it as a full application with a user interface or as individual components.

Either way, RealReflect is likely to result in a new generation of virtual reality, one that is more realistic than ever before.