VR motion programming.

[MetalOS]

Hello everyone. I told you I would ask a lot of questions...

Now that I have roughly understood how Ultra Engine and VR work, I am looking to move around in the 3D world that I have created. As at the moment I don't know anything about Lua and I'm in the process of learning it, I searched in the Ultra Engine documentation and on various websites that deal with Lua and VR but I'm stuck on my code. If someone would be kind enough to look at my code and tell me what's wrong it would help me make progress. Thanks in advance.

-- Initialize Steam (optional)
if not Steamworks.Initialize() then return 1 end

-- Load FreeImage plugin (optional)
local fiplugin = LoadPlugin("Plugins/FITextureLoader")

-- Get the displays
local displays = GetDisplays()

-- Create a window
local window = CreateWindow("Ultra Engine", 0, 0, 1280, 720, displays[1], WINDOW_CENTER | WINDOW_TITLEBAR)

-- Create a framebuffer
local framebuffer = CreateFramebuffer(window)

-- Create a world
local world = CreateWorld()

-- Get the VR headset (HMD)
local hmd = GetHmd(world, true)

-- Load a map
local mapname = "Maps/start.ultra"
local cl = CommandLine()
if type(cl["map"]) == "string" then mapname = cl["map"] end
local scene = LoadMap(world, mapname)

-- Initialize player's position and rotation offsets
local player_x_offset = 0
local player_y_offset = 0
local player_z_offset = 0
local player_yaw_offset = 0

-- Movement and rotation speed
local move_speed = 0.1
local rotation_speed = 2

-- Main loop
while window:KeyDown(KEY_ESCAPE) == false and window:Closed() == false do
    
    -- Garbage collection step
    collectgarbage()

    -- Update the world
    world:Update()
    Steamworks.Update()

    -- Check input from the VR controllers
    if hmd:ButtonDown(VR_LEFT_TOUCHPAD) then
        -- Get the forward direction from the left controller
        local forward = hmd:GetControllerDirection(LEFT_HAND)
        player_x_offset = player_x_offset + forward.x * move_speed
        player_z_offset = player_z_offset + forward.z * move_speed
    end

    if hmd:ButtonDown(VR_RIGHT_TOUCHPAD) then
        -- Get the yaw (horizontal rotation) from the right controller
        local yaw_rotation = hmd:GetControllerRotation(LEFT_HAND).y
        player_yaw_offset = player_yaw_offset + yaw_rotation * rotation_speed
    end

    -- Apply the offset to the player using SetOffset
    hmd:SetOffset(player_x_offset, player_y_offset, player_z_offset)
    hmd:SetRotation(0, player_yaw_offset, 0)

    -- Render the world to the framebuffer
    world:Render(framebuffer)
end

-- Shutdown Steamworks
Steamworks.Shutdown()

 

[Dreikblack]

Half of vr related code lines are not even from Ultra API

I don't have VR device to test examples but here some tips:

https://www.ultraengine.com/learn/Hmd?lang=lua - VR display, you can set position and rotation with:

 

    local position = Vec3(0, 0, 0)

    local rotation = Vec3(0, 0, 0)

    hmd:SetOffset(position, rotation)

 

Controllers can be got from display, their api https://www.ultraengine.com/learn/VrController?lang=lua

 

    local leftController = hmd.controllers[0]

    local rightController = hmd.controllers[1]

    local forward = leftController:GetAxis(VRAXIS_TOUCHPAD)

 

[MetalOS]

Tank's Dreikblack,

I'm new to programming and I tried to mix the different codes I found on Lua to use the controllers on my headset. I will use your feedback to try to improve this.

 

[Josh]

The Hmd class is your starting point:
https://www.ultraengine.com/learn/Hmd

Simply retrieving the HMD will initialize VR.

The Hmd:SetOffset method handles translation and rotation:
https://www.ultraengine.com/learn/Hmd_SetOffset

The Hmd class also provides access to your controllers:
https://www.ultraengine.com/learn/VrController

Keep in mind that either of the controller variables might be NULL until your system starts detecting the hardware.

[MetalOS]

Thanks for the clarification Josh. I'll watch this after work

[MetalOS]

I still have errors on the controllers. Afterwards I'm not sure of my syntax, I'm going a bit blind.

-- Initialiser Steam (optionnel)
if not Steamworks.Initialize() then return 1 end

-- Charger le plugin FreeImage (optionnel)
local fiplugin = LoadPlugin("Plugins/FITextureLoader")

-- Obtenir les écrans
local displays = GetDisplays()

-- Créer une fenêtre
local window = CreateWindow("Ultra Engine", 0, 0, 1280, 720, displays[1], WINDOW_CENTER | WINDOW_TITLEBAR)

-- Créer un framebuffer
local framebuffer = CreateFramebuffer(window)

-- Créer un monde
local world = CreateWorld()

-- Récupérer le casque VR (HMD)
local hmd = GetHmd(world, true)

-- Charger une carte
local mapname = "Maps/start.ultra"
local cl = CommandLine()
if type(cl["map"]) == "string" then mapname = cl["map"] end
local scene = LoadMap(world, mapname)

-- Initialiser la position et la rotation du joueur
local position = Vec3(0, 0, 0)
local rotation = Vec3(0, 0, 0)

-- Récupérer les contrôleurs gauche et droit depuis la propriété controllers
local leftController = hmd.controllers[1]  -- Généralement l'index 1 est le contrôleur gauche
local rightController = hmd.controllers[2] -- Généralement l'index 2 est le contrôleur droit

-- Vitesse de déplacement et de rotation
local move_speed = 0.1
local rotation_speed = 2

-- Boucle principale
while window:KeyDown(KEY_ESCAPE) == false and window:Closed() == false do
    
    -- Collecte des déchets
    collectgarbage()

    -- Mettre à jour le monde
    world:Update()
    Steamworks.Update()

    -- Vérifier l'entrée du joystick ou pavé tactile du contrôleur gauche pour le déplacement
    if leftController then
        local axisLeft = leftController:GetAxis(0) -- Récupérer l'axe du joystick/pavé tactile (axis 0)
        position.x = position.x + axisLeft.x * move_speed
        position.z = position.z + axisLeft.y * move_speed
    end

    -- Vérifier l'entrée du joystick ou pavé tactile du contrôleur droit pour la rotation
    if rightController then
        local axisRight = rightController:GetAxis(0) -- Récupérer l'axe du joystick/pavé tactile (axis 0)
        rotation.y = rotation.y + axisRight.x * rotation_speed
    end

    -- Appliquer la position et la rotation au joueur
    hmd:SetOffset(position, rotation)

    -- Rendre le monde dans le framebuffer
    world:Render(framebuffer)
end

-- Arrêter Steamworks
Steamworks.Shutdown()

 

[Josh]

Like I said, hmd.controllers[1] and [2] might (probably will) be nil when they are first called, because the VR system has not initialized and detected your hardware yet. Do not put these values into a variable. Instead, use the values directly and don't ever assume they are they. They can even disappear if the controller gets turned off or runs out of batteries.

[MetalOS]

Sorry Josh but does not understand and I think that I am in Trein to mix my brush. Do you have a simple example to move in a VR world. To understand how it works?

[Josh]

I am compiling a new build right now. I will see about this after I finish uploading the update.

[Josh]

Okay, first thing tomorrow I will code a sample that shows teleportation locomotion and paste it in the documentation.

[MetalOS]

Great thanks Josh.

[Josh]

Okay, I made two mistakes.

First, I had the default value for the initialize parameter in GetHmd set to false. 

Second, the way in which the Hmd.controllers array was exposed to Lua was done incorrectly.

The fix for both of these is up on the beta branch now. You can sync your project to get the new Lua executables.

A working example is here:
https://github.com/UltraEngine/Documentation/blob/master/Lua/VrController_ButtonHit.md#example

Now that controller input is working correctly I think you will have an easier time. Please let me know if you have any more questions.

[MetalOS]

Hi Josh you are a boss.

I'll check it out when I get home and let you know. Thanks again.

[MetalOS]

Hi Josh, sorry I didn't have much time to try but now the 3d world is displayed in my Quest3. When I have time I will try to move the camera with the joysticks to see.

[Josh]

[MetalOS]

Is there a way to fix the world in VR? Because for the moment what I have created is displayed well in my Quest 3 but it is very jerky. For my tests I use the Nature DLC. Maybe objects with fewer polygons?

[Josh]

Are you saying that the framerate is slow?

[MetalOS]

Yes I must be at 10 or 15 fps maximum

[Josh]

What graphics card do you have? Can you post a screenshot of the scene you are rendering?

[MetalOS]

I'm using an Nvidia GTX 1660Ti. For the scene I'm at work and I couldn't post a capture at the moment but I will do that once I get home. I created a 512x512 scene with grassy soil, a dozen rocks and around 200 fir trees.

[MetalOS]

Here are the screenshots of my scene.

 

 

[Josh]

There's a lot you can do to adjust the speed in the foliage layer properties:

• The imposter distance can be used to show a billboard for trees that are further away.
• Shadows can be disabled on grass and small plants.
• The view distance of grass and plants can be made lower.

[MetalOS]

Ok thanks Josh. I'm going to look to optimize everything, maybe deactivate certain shadows which are not necessary.