Stimulus preparation

The system architecture and methodology described here introduce a novel two-stage carrier signal generation approach:  first, proven psychophysical carrier signals are adapted to the environment and second, the adapted carriers are animated along contextual motion paths.  After stimulus preparation, motion-modulated, adapted carrier signals integrate into a dynamic environment to augment observer experiences.  (See related projects here and here; further dissertation research forthcoming)

The novel series of stimuli used throughout the experimentation are generated once the raw environmental footage is stabilized and formatted for the study environment.  All stimuli are adapted according to the following process:

  • Regions of interest (ROIs) are identified within the footage to serve as sources of scene data.

 

  • Scene parameters of hue, saturation, value, spatial frequency and optical flow are calculated within the ROIs

 

  • Carrier signal locations are identified on the simulated environment within the observer field of view.

 

  • Carrier signal envelopes are generated (either traditional Gaussian windows or scene-adaptive, non-uniform envelopes, measured in arc min)

 

  • Raw psychophysical carrier signals (such as first-and second-order Gabor patches) are generated.

 

  • Carrier signal motion is determined (either from scene data or codex paths)

 

  • Adaptive carrier signals are generated from scene data and motion parameters.

 

  • Adaptive carrier signals are integrated at the defined carrier signal locations within the simulated environment.
Click to enlarge

For the studies conducted to validate this system throughput, we utilized sinusoidal gratings and Gaussian white noise, generated in MATLAB 2019b.  We also built in a new functionality to use ROIs from natural sources as psychophysical structures.  Image regions are chosen for their spatial frequency characteristics, favoring ROIs with minimal artifacts.  While outside the scope of this dissertation, future work intends to fully parameterize and evaluate the use of complex natural sources for eliciting perceived peripheral motion shifts.

Click to view process images for biological structure preparation for Fourier analysis (envelope compatibility)

Chromatic adaptability of the gaussian windowed noise.  In the three figures, the panel on the right shows a section of the simulated environment while an enhanced detail panel on the left highlights variation in the patches calculated from the mean HSV values within the frame at time t = {t_1, t_2, t_3}. 

Sample instantiation of viewing environment
Scene HSV values at time t = {t1, t2, t3}

In this instantiation of the display environment, the carrier signal adaptation is accomplished by using an overlay blending mode, a variation of element-wise multiplication of the two source images which results in a linear interpolation of the top layer between the span of values in the base layer:

 

In other cases, the raw psychophysical structures were generated in MATLAB frame by frame by directly editing the HSV values to match the source ROI.  As the three chromatic adaptations above demonstrate, there is a significant variation in the HSV values within the scene.  These variations are summarized in the figure to the left, with an unadapted baseline for reference.

[Full posting forthcoming]

© 2020 M. Everett Lawson.

error: Copyright M. Everett Lawson 2020