01 / 22
Università degli Studi di BergamoMETRO_ART · Movement Experience through Rhythmic Organization in Audiovisual Representational Texts
NECS 2026 Conference • Montpellier
Immersive Rhythms
The Invisible Forms of Movement Perception in Virtual Reality
SpeakersAdriano D'Aloia
Anja Boato
InstitutionUniversità degli studi di Bergamo
Date19 June 2026
Project funding and Università degli Studi di Bergamo logos
Theoretical frame

Rhythm is not only temporal:
it is perceptual organization

In audiovisual experience, rhythm emerges from the way events become ordered, anticipated and felt as a patterned flow. (Gibson, 1979; Noë, 2004)

Virtual reality makes this organization spatial: rhythm is perceived through duration, direction, proximity and bodily orientation.

From sequence to experience

A gesture is not perceived as a neutral series of movements. It is read as a dynamic configuration of accents, continuities and pauses that guides attention and expectation.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Theoretical frame

In VR, rhythm becomes
a question of presence

Spatial anchoring

Where rhythm happens

The viewer is not only looking at movement, but is located in relation to it: distance, scale and orientation become part of the perceptual field.

(Steuer, 1992; Slater, 2009)
Embodied attention

How rhythm is followed

Head and eye movements can reorganize salience, making rhythm depend on what the viewer chooses or is able to track.

(Witmer & Singer, 1998; Riva, Waterworth & Waterworth, 2004)
Affective intensity

How rhythm is felt

Immersion can increase involvement, but it can also introduce fatigue, discomfort or distraction that weakens rhythmic sensitivity.

(Slater, 2009; Bailenson, 2018)
Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Theoretical frame

VR does not automatically
make rhythm more intense

The decisive variable is agency

When the viewer can explore, choose a viewpoint or act inside the environment, rhythm may become an embodied coordination between gesture and user movement.

(Bolter & Grusin, 1999; Slater, 2009; Latoschik & Wienrich, 2022)

When VR reproduces the same frontal, non-interactive exposure as a cinema "flat" screen, immersion may remain mostly optical rather than performative. (Steuer, 1992; Slater, 2009)

  • Low agency: rhythm remains close to video viewing.
  • High agency: rhythm may become navigational and bodily.
  • Comfort constraints: device weight and fatigue can counteract presence.
Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Relation to Milan experiment

From Milan’s validation
to the Bergamo VR/Cinema extension

Shared ground

Same rhythmic grammar

Both experiments work from combinations of complexity, direction and movement quality, asking whether structural differences become perceptible as regularity, fluidity and energy.

Main difference

The medium becomes experimental

The Bergamo study removes performer gender and one spatial axis, then adds the Cinema/VR comparison and eye-tracking to test how mediation reshapes perception.

Interpretive bridge

From validation to modulation

Milan's experiment validates the perceptual grammar of gesture. The Bergamo extension asks whether that grammar is stable across media or modulated by immersive viewing. Although the validation study conducted in Milan identified less significant trends, our results are consistent with the perception that regularity and fluidity are associated with simpler movement patterns.

Working hypothesis

If VR remains non-interactive and visually similar to flat viewing, differences should be localized rather than global.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
The research move

From validated gestures
to media-specific perception

The Bergamo unit extends the original validation paradigm by asking whether the same rhythmic structure is perceived differently when experienced on a cinema "flat" screen or in immersive VR.

Continuity

Movement grammar

Sequences still vary in complexity, direction and quality: accented gestures, sustained gestures and their ordered combinations.

Reduction

Simpler design

Performer gender and the opening-closing axis are removed, concentrating the experiment on vertical/horizontal avatar movement.

Extension

Medium as variable

Each stimulus is presented in both cinema "flat" screen and VR conditions while eye-tracking data and self-report ratings are collected.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Stimulus construction

A compact factorial design
for avatar movement

Articulation

Simple/Complex

Minimal units versus more articulated units, with identical clip duration.

Direction

Horizontal/Vertical

Three spatial axes, each expressed in two directions: horizontal (right-left/left-right) and vertical (ascending/descending).

Sequence

Accented/Sustained

Ordered combinations of accented movements (abrupt, fast and energetic) and sustained movements (smooth, regular and continuous).

Medium

Cinema/VR

The same clips are experienced on a standard screen and in immersive display.

Main hypothesis

The reduced design preserves the rhythmic variables while isolating the perceptual contribution of the viewing medium: complexity, direction, quality of movement.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Movement units

Simple and complex patterns
share the same duration

Simple movement stimuli

8 clips

2 directions x 2 sequences for each direction x 2 movement qualities.

  • Lower motor density
  • More direct quality contrast
  • Predictability concentrated in two movements
Complex movement stimuli

8 clips

2 directions x 2 sequences for each direction x 2 combinations of quality.

  • Higher motor density
  • Alternation and cyclicity
  • Greater internal articulation

Total exposure

16 clips × 3 repetitions across Cinema and VR conditions, with repeated VAS ratings for Energy, Fluidity and Regularity.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Experimental protocol

Four phases,
one repeated perceptual task

Phase 1

Preliminary profiling

Questionnaires assess each participant's propensity for immersion.

  • ITQ
  • VMIQ-2 EVI
  • VMIQ-2 IVI
  • VMIQ-2 KIN
Phase 2

Randomized clips

Stimuli are viewed under Cinema and VR conditions. Each clip is repeated three times. Oculometric measurement is also recorded during stimulus exposure.

Phase 3

VAS ratings

After each presentation, participants rate regularity, fluidity or energy from 0 to 100.

Phase 4

VR immersion

Final questionnaires capture the subjective sense of immersion in the VR environment.

  • F-IEQ
  • PQ
Sample recruitment

30 / 55 subjects

30
55 target sample
Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Stimuli production

Neutral avatars,
distraction-free environments

Movements are performed by neutral avatars inside a neutral environment and reconstructed from kinematic data derived from the female dancer in the first validation study.

Technical setup

iMotions supports eye-tracking acquisition across Cinema and VR, while the VR condition uses a Varjo XR-4 head-mounted display.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Model evidence

Movement quality and direction
organize perceptual judgments

.462
Conditional R²
Model including random subject variance; marginal R² = .138.
80.18
Question × Movement Quality F
The strongest interaction in the model, p < .001.
56.69
Direction F
Direction remains highly significant, p < .001.
5.29
Condition F
Cinema/VR is significant but small, p = .022.

Interpretive point

The model shows that the medium remains a small main effect, while the perceptual grammar is organized mainly by movement quality, question and direction.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Question × Complexity

Complexity modulates
energy, fluidity and regularity

Interaction Question × Complexity 0 20 40 60 80 Estimated marginal mean Energy Fluidity Regularity Question Simple Complex 52.461.462.156.365.760.5

Perceptual profile: Question × Complexity is significant (F = 3.02, p = .049). Complexity increases energy, while simple patterns are associated with higher fluidity and regularity.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Question × Movement Type

Movement type reorganizes
energy, fluidity and regularity

Interaction Question × Movement Type 0 20 40 60 80 Estimated marginal mean Energy Fluidity Regularity Question 64.861.861.040.055.055.856.769.260.060.160.971.4
Accented–Accented
Accented–Sustained / Accented–Sustained
Sustained–Accented / Sustained–Accented
Sustained–Sustained

Perceptual profile: Question × Movement Type is the dominant interaction (F = 80.18, p < .001).

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Direction × Movement Type

Direction modulates
the perceptual force of movement type

Interaction Direction × Movement Type 0 20 40 60 80 Estimated marginal mean Horizontal Vertical Direction 56.656.155.859.861.963.064.060.6
Accented–Accented
Accented–Sustained / Accented–Sustained
Sustained–Accented / Sustained–Accented
Sustained–Sustained

Perceptual profile: Direction × Movement Type is significant (F = 7.25, p < .001).

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Question × Complexity × Direction × Movement Type

Interaction of movement type
across complexity and direction

Interaction Question × Complexity × Direction × Movement Type
Complex · Horizontal 0 40 80 Energy Fluidity Regularity 59.155.953.753.257.159.3
Complex · Vertical 0 40 80 Energy Fluidity Regularity 64.666.258.060.263.162.6
Simple · Horizontal 0 40 80 Energy Fluidity Regularity 59.740.051.368.856.670.7
Simple · Vertical 0 40 80 Energy Fluidity Regularity 69.940.058.769.763.572.1
Accented–AccentedAccented–Sustained / Accented–SustainedSustained–Accented / Sustained–AccentedSustained–Sustained

Four-way view: each facet follows the same two-bar grouping used in slide 16: mixed movement types in Complex, pure movement types in Simple.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Condition × Question × Complexity × Direction

Cinema and VR across
complexity and direction

Interaction Condition × Question × Complexity × Direction
Complex · Horizontal04080EnergyFluidityRegularity59.056.054.051.059.058.0
Complex · Vertical04080EnergyFluidityRegularity65.065.062.056.064.061.0
Simple · Horizontal04080EnergyFluidityRegularity51.048.060.061.063.065.0
Simple · Vertical04080EnergyFluidityRegularity57.053.065.064.067.068.0
CinemaVR

Medium profile: the plot compares Cinema and VR within each Complexity × Direction combination, highlighting localized rather than global medium effects.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Main analytical results

What the analysis shows
about rhythm perception

01

Sustained sequences drive fluidity and regularity

Sequences composed only of sustained movements are rated much higher for regularity and fluidity. This is coherent with the Milan results, but the difference from accented combinations appears more marked in the Bergamo data.

02

Accented sequences drive energy

Energy is rewarded especially in fully accented sequences. This confirms the expected perceptual association between accented movement, abruptness, speed and energetic intensity.

03

Sequence is the most coherent variable

Because the Bergamo clips are built from only four validated sequences, it is preferable to describe the factor as sequence rather than isolated movement quality, while still acknowledging the perceptual weight of accented vs sustained movement.

04

The medium has limited impact

Cinema/VR differences remain localized and small. The main perceptual structure is generated by movement sequence and direction rather than by a global medium effect.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Preliminary conclusions

Why Cinema and VR
remain close

Factor 1

Medium differences are limited

Cinema is only slightly higher than VR (+1.62, p = .022). Differences between the two viewing conditions remain limited in magnitude.

Factor 2

Rhythmic structure dominates

Direction (F = 56.69) and Question × Movement Quality (F = 80.18) explain the perceptual organization more strongly than the viewing device.

Implication. In this non-interactive setup, VR does not generate a global perceptual advantage. The strongest perceptual grammar is produced by direction and movement quality.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Closing

The experiment supports
a perceptual grammar of movement

The Bergamo results reinforce the METRO_ART framework: rhythm perception is systematic, and it is organized primarily by movement quality and direction rather than by a global Cinema/VR split.

For future analysis

Eye-tracking data are recorded simultaneously in both viewing conditions.

Next analytical direction

Integrate VAS ratings with eye-tracking and immersion questionnaires to test whether attention, comfort and subjective presence explain the localized Cinema/VR differences.

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Bibliography

References

Bailenson, J. (2018). Experience on Demand: What Virtual Reality Is, How It Works, and What It Can Do. W. W. Norton.
Bolter, J. D., & Grusin, R. (1999). Remediation: Understanding New Media. MIT Press.
Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Houghton Mifflin.
Latoschik, M. E., & Wienrich, C. (2022). Congruence and plausibility, not presence? Frontiers in Virtual Reality.
Noë, A. (2004). Action in Perception. MIT Press.
Riva, G., Waterworth, J. A., & Waterworth, E. L. (2004). The layers of presence. CyberPsychology & Behavior, 7(4), 402–416.
Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B, 364(1535), 3549–3557.
Steuer, J. (1992). Defining virtual reality: Dimensions determining telepresence. Journal of Communication, 42(4), 73–93.
Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence, 7(3), 225–240.
Cameron, D., & Grahn, J. A. (2020). Perception of Rhythm. In R. McClelland & R. Hartenberger (Eds.), The Cambridge Companion to Rhythm. Cambridge: Cambridge University Press.
Dooley, K. (2017). Storytelling with virtual reality in 360-degrees: A new screen grammar. Studies in Australasian Cinema, 11(3), 161–171.
Serrano, A., Sitzmann, V., Ruiz-Borau, J., Wetzstein, G., Gutierrez, D., & Masia, B. (2017). Movie Editing and Cognitive Event Segmentation in Virtual Reality Video. ACM Transactions on Graphics, 36(4).
Zou, Q., Zheng, W., Su, Z., Zhang, L., Zhuo, Z., & Li, D. (2025). The Neural Impact of Editing on Viewer Narrative Cognition in Virtual Reality Films: Eye-Tracking Insights into Neural Mechanisms. Frontiers in Psychology, 16.
Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
Università degli Studi di Bergamo

Research Team

Adriano D'Aloia

Adriano D'Aloia

Anja Boato

Anja Boato

Nunzio Langiulli

Nunzio Langiulli

Metro_Art • Movement Experience Through Rhythmic Organization in Audiovisual Representational Texts
Project funding and Università degli Studi di Bergamo logos
NECS 2026 CONFERENCE • MONTPELLIER
QR code www.ritmi.net
www.ritmi.net
THE INVISIBLE RHYTHM