There are a number of applications in computer graphics that require as a first step the accurate estimation of principal direction ectors at arbitrary vertices on a triangulated surface. Although several methods for calculating principal directions over such models have been previously proposed, we have found in practice that all exhibit unexplained large errors in some cases. In this paper we describe our theoretical and experimental investigations into possible sources of errors in the approximation of principal direction vectors from triangular meshes, and suggest a new method for estimating principal directions that can yield better results under some circumstances.

In this paper we describe the results of two comprehensive controlled observer experiments intended to yield insight into the following question: If we could design the ideal texture pattern to apply to an arbitrary smoothly curving surface in order to enable its 3D shape to be most accurately and effectively perceived, what would the characteristics of that texture pattern be?
       We begin by reviewing the results of our initial study in this series, which were presented at the 2003 IEEE Symposium on Information Visualization, and offer an expanded analysis of those findings. We continue by presenting the results of a follow-on study, in which we sought to more specifically investigate the separate and combined influences on shape perception of particular texture components, with the goal of obtaining a clearer view of their potential information carrying capacities. In each study we investigated observers' ability to identify the intrinsic shape category of a surface patch (elliptical, hyperbolic, cylindrical or flat) and its extrinsic surface orientation (convex, concave, both or neither).
       In our first study we compared performance under 8 different texture type conditions, plus 2 projection conditions (perspective or orthographic) and 2 viewing conditions (head-on or oblique). In this study we found that: 1) shape perception was better facilitated, in general, by the bi-directional 'principal direction grid' pattern than by any of the 7 other patterns tested; 2) shape type classification accuracy remained high under the orthographic projection condition for some texture types when the viewpoint was oblique; 3) perspective projection was required for accurate surface orientation classification; and 4) shape classification accuracy was higher when the surface patches were oriented at a (generic) oblique angle to the line of sight than when they were oriented (in a non-generic pose) to face the viewpoint straight-on.
       In our second study, we compared performance under 8 new texture type conditions, redesigned to facilitate gathering insight into the cumulative effects of specific individual directional components in a wider variety of multi-directional texture patterns. In this follow- on study we found that shape classification accuracy was equivalently good under a variety of test patterns that included components following either the first or first and second principal directions, in addition to other directions, suggesting that a principal direction grid texture is not the only possible 'best option' for enhancing shape representation.

Studies have shown that although shading can give an image a "natural" look and is an important shape cue, visual perception of surface shape from shading only is severely limited when the surface is viewed locally without other visual cues such as occluding contours [Mamassian and Kersten 1996; Erens et al. 1993]. Research has shown that when the "right" texture is added to the surface, observers can reliably infer the 3D structure of the underlying shape. In our own previous work, we have found that the performance of subjects' shape judgment is significantly better with principal direction oriented texture pattern than other directional texture following either a uniformly constant direction or varying non-geodesic paths unrelated to the surface geometry. In this paper, we report our findings of a new study further investigating the effect of anisotropic textures on shape perception when the surface texture is represented in the form of a pattern of luminance variations as well as of surface relief variations. We predicted that 1) performance would be higher with relief textures than luminance textures, and that 2) the performance would be poorer with the anisotropic textures that do not follow principal directions. The results confirmed both of our predictions.

Considerable evidence suggests that a viewer's perception of the 3D shape of a polygonally-defined object can be significantly affected (either masked or enhanced) by the presence of a surface texture pattern. However, investigations into the specific mechanisms of texture's effect on shape perception are still ongoing and the question of how to design and apply a texture pattern to a surface in order to best facilitate shape perception remains open. Recently, we have suggested that, for anisotropic texture patterns, the accuracy of shape judgments may be significantly affected by the orientation of the surface texture pattern anisotropy with respect to the principal directions of curvature over the surface. However, it has been difficult, until this time, to conduct controlled studies specifically investigating the effect of texture orientation on shape perception because there has been no simple and reliable method for texturing an arbitrary doubly curved surface with a specified input pattern such that the dominant orientation of the pattern everywhere follows a predefined directional vector field over the surface, while seams and projective distortion of the pattern are avoided.
        In this paper, we present a straightforward and highly efficient method for achieving such a texture and describe how it can potentially be used to enhance shape representation. Specifically, we describe a novel, efficient, automatic algorithm for seamlessly synthesizing, from a sample 2D pattern, a high resolution fitted surface texture in which the dominant orientation of the pattern locally follows a specified vector field over the surface at a per-pixel level and in which seams, projective distortion, and repetition artifacts in the texture pattern are nearly completely avoided. We demonstrate the robustness of our method with a variety of texture swatches applied to standard graphics data sets and we explain how our method can be used to facilitate research in the perception of shape from texture.

In scientific visualization, there are many applications in which researchers need to achieve an integrated understanding of the three-dimensional shapes and relative depth distances of multiple overlapping objects. Transparent surface rendering has the potential to be a useful device, in many of these cases, for simultaneously portraying multiple superimposed structures in a single image, so that their complex spatial relationships can be more accurately and comprehensively inferred. The challenge is to determine how to portray the outermost objects in such a way that they can be both effectively seen, and also seen through, at the same time. In this article we briefly review a variety of issues in transparent surface rendering, and describe some novel graphical techniques that aim to enhance the representation of external transparent surfaces, while maintaining the visibility of internal structures, through the careful design and application of sparsely distributed opaque surface markings. Integral to this discussion is an analysis of the effects that various texture pattern characteristics, such as orientation, have on surface shape perception.

Studies have shown that observers' judgments of surface orientation and curvature are affected by the presence of surface texture pattern. However, the question of designing a texture pattern that does not hide the surface information nor conveys a misrepresentation of the surface remains unsolved. The answer to this question has important potential impact across a wide range of visualization applications. Molecular modeling and radiation therapy are among the many fields that are in need of accurately visualizing their data that could benefit from such methods.

If we could design the perfect texture pattern to apply to any smooth surface in order to enable observers to more accurately perceive the surface's shape in a static monocular image taken from an arbitrary generic viewpoint under standard lighting conditions, what would the characteristics of that texture pattern be?
      In order to gain insight into this question, our group has developed an efficient algorithm for synthesizing a high resolution texture pattern, derived from a provided 2D sample, over an arbitrary doubly curved surface in such a way that the orientation of the texture is constrained to follow a specified underlying vector field over the surface, at a per-pixel level, without evidence of seams or projective distortion artifacts.
       In this paper, we report the findings of a recent experiment in which we attempt to use this new texture synthesis method to assess the shape information carrying capacity of two different types of directional texture patterns (unidirectional and bi-directional) under three different orientation conditions (following the first principal direction, following a constant uniform direction, or swirling sinusoidally in the surface). In a four alternative forced choice task, we asked participants to identify the quadrant in which two B-spline surfaces, illuminated from different random directions and simultaneously and persistently displayed, differed in their shapes.
       We found, after all subjects had gained sufficient training in the task, that accuracy increased fairly consistently with increasing magnitude of surface shape disparity, but that the characteristics of this increase differed under the different texture orientation conditions. Subjects were able to more reliably perceive smaller shape differences when the surfaces were textured with a pattern whose orientation followed one of the principal directions than when the surfaces were textured with a pattern that either gradually swirled in the surface or followed a constant uniform direction in the tangent plane regardless of the surface shape characteristics. These findings appear to support our hypothesis that anisotropic textures aligned with the first principal direction may facilitate shape perception, for a generic view, by making more, reliable information about the extent of the surface curvature explicitly available to the observer than would be available if the texture pattern were oriented in any other way.

Perception of the 3D shape of a smoothly curving surface can be facilitated or impeded by the use of different surface texture patterns. In this paper we report the results of a series of experiments intended to provide insight into how to select or design an appropriate texture for shape representation in computer graphics. In these experiments, we examine the effect of the presence and direction of luminance texture pattern anisotropy on the accuracy of observers' judgments of 3D surface shape. Our stimuli consist of complicated, smoothly curving level surfaces from a typical volumetric dataset, across which we have generated four different texture patterns via 3D line integral convolution: one isotropic and three anisotropic, with the anisotropic patterns oriented across the surface either in a single uniform direction, in a coherently varying direction, or in the first principal direction at every surface point. Observers indicated shape judgments via manipulating an array of local probes so that their circular bases appeared to lie in the tangent plane to the surface at the probe's center, and the perpendicular extensions appeared to point in the direction of the local surface normal. Stimuli were displayed as binocularly viewed flat images in the first trials, and in stereo during the second trials. Under flat viewing, performance was found to be better in the cases of the isotropic pattern and the anisotropic pattern that followed the first principal direction than in the cases of the other two anisotropic patterns. Under stereo viewing, accuracy increased for all texture types, but was still greater for the isotropic and principal direction patterns than for the other two. Our results are consistent with a hypothesis that texture pattern anisotropy impedes surface shape perception in the case that the direction of the anisotropy does not locally follow the direction of greatest normal curvature.

While many factors contribute to shape perception, psychological research indicates that the direction of lines on the surface may have an important influence. This is especially the case when other techniques (shading, silhouetting) do not present sufficient shape information. The psychology literature suggests that lines in the principal directions of curvature may communicate surface shape better than lines in other directions. Moreover, principal directions have the quality of geometric invariance so line directions are based on the surface geometry and are viewpoint and light source independent, and the lines do not move above over the surface during animation unless desired. In this work we describe principal direction line drawings which show the flow of curvature over the surface. The technique is presented for arbitrary surfaces represented by either 3D volume data or a polygonal surface mesh. The latter format is common in the field of computer graphics yet thus far has not been widely used for principal direction estimation. The methods offered in this paper can be used alone or in conjunction with other NPR techniques to improve artistic 3D renderings of arbitrary surfaces.

This paper describes how the set of principal directions and principal curvatures can be understood to define a natural "flow" over the surface of an object and, as such, can be used to guide the placement of the lines of a stroke texture that seeks to represent 3D shape in a perceptually intuitive way. The driving application for this work is the visualization of layered isovalue surfaces in volume data, where the particular identity of an individual surface is not generally known a priori and observers will typically wish to view a variety of different level surfaces from the same distribution, superimposed over underlying opaque structures.
       This paper describes how, by advecting an evenly distributed set of tiny opaque particles, and the empty space between them, via 3D line integral convolution through the vector field defined by the principal directions and principal curvatures of the level surfaces passing through each gridpoint of a 3D volume, it is possible to generate a single scan-converted solid stroke texture that can be used to illustrate the essential shape information of any level surface in the data. By redefining the length of the filter kernel according to the magnitude of the maximum principal curvature of the level surface at each point around which the convolution is applied, one can generate longer strokes over more the highly curved areas, where the directional information is both most stable and most relevant, and at the same time downplay the visual impact of the directional information indicated by the stroke texture in the flatter regions. In a voxel-based approach such as this one, stroke narrowness will be constrained by the resolution of the volume within which the texture is represented. However, by adaptively indexing into multiple pre-computed texture volumes, obtained by advecting particles of increasing sizes, one may selectively widen the strokes at any point by a variable amount, determined at the time of rendering, to reflect shading information or any other function defined over the volume data.

Transparency can be a useful device for depicting multiple overlapping surfaces in a single image. The challenge is to render the transparent surfaces in such a way that their three-dimensional shape can be readily understood and their depth distance from underlying structures clearly perceived.
       This paper describes our investigations into the use of sparsely-distributed discrete, opaque texture as an "artistic device" for more explicitly indicating the relative depth of a transparent surface and for communicating the essential features of its 3D shape in an intuitively meaningful and minimally occluding way. The driving application for this work is the visualization of layered surfaces in radiation therapy treatment planning data, and the technique is illustrated on transparent isointensity surfaces of radiation dose. We describe the perceptual motivation and artistic inspiration for defining a stroke texture that is locally oriented in the direction of greatest normal curvature (and in which individual strokes are of a length proportional to the magnitude of the curvature in the direction they indicate), and discuss two alternative methods for applying this texture to isointensity surfaces defined in a volume. We propose an experimental paradigm for objectively measuring observers' ability to judge the shape and depth of a layered transparent surface, in the course of a task relevant to the needs of radiotherapy treatment planning, and use this paradigm to evaluate the practical effectiveness of our approach through a controlled observer experiment based on images generated from actual clinical data.