Node.js and webpack deployment of GeometryViewer with AR

Lois_Vazquez · January 17, 2023, 4:30pm

Hello.

I’m new to vtk.js so I apologize if I’m skipping some basic point that I don’t know. I am trying to deploy the GeometryViewer example with node.js and webpack in order to test the augmented reality support. My dependencies and building scripts are:

   "scripts": {
     "dev": "node_modules/.bin/webpack",
     "server": "./node_modules/.bin/http-server dist -p 3007 -c-1",
   },
   "dependencies": {
     "@kitware/vtk.js": "^25.15.1"
   },
   "devDependencies": {
     "@babel/preset-env": "^7.13.12",
     "autoprefixer": "^10.2.5",
     "babel-loader": "^8.2.2",
     "buffer": "^6.0.3",
     "css-loader": "^6.7.3",
     "html-loader": "^2.1.2",
     "http-server": "^14.1.1",
     "postcss-loader": "^7.0.2",
     "readable-stream": "^2.3.7",
     "stream": "^0.0.2",
     "stream-browserify": "^3.0.0",
     "style-loader": "^3.3.1",
     "webpack": "^5.27.2",
     "webpack-cli": "^4.5.0",
     "webpack-dev-server": "^3.11.2"
   },

From a PC, both your example and your code deployed on my localhost work fine (upper image is my localhost, lower one is your example):

In my mobile phone, your example works well:

However, if I access my deployed code from mobile, the same code does not recognize my device as XR/AR/VR capable:

To check compatibility, I use the following code:

var isXRCapable = navigator.xr !== undefined && fullScreenRenderWindow.getApiSpecificRenderWindow().getXrSupported();
var isARCapable = navigator.xr !== undefined && fullScreenRenderWindow.getApiSpecificRenderWindow().getXrSupported() && navigator.xr.isSessionSupported('immersive-ar');
var isVRCapable = navigator.xr !== undefined && fullScreenRenderWindow.getApiSpecificRenderWindow().getXrSupported() && navigator.xr.isSessionSupported('immersive-vr');
if (!isXRCapable) {// XR capable
  window.alert("Your device is not XR capable");
}
if (!isARCapable) {  // AR capable
  window.alert("Your device is not AR capable");
}
if (!isVRCapable) {  // VR capable
  window.alert("Your device is not VR capable");
}

What could be going wrong? Should I try some other way of deploying the example?Since the code is the same as the example, I understand that I’m missing some additional configuration to make this code work, but I can’t find any references to that.

Thanks in advance.

tbirdso · January 17, 2023, 6:57pm

Hi @Lois_Vazquez , thanks for using vtk.js!

It would be helpful to have more information on what is happening. Could you please add the following details:

What mobile device you are using
What browser you are using

Also, what is the output that you see from the following checks?

navigator.xr !== undefined

navigator.xr.isSessionSupported('immersive-ar')

fullScreenRenderWindow.getApiSpecificRenderWindow().getXrSupported()

tbirdso · January 17, 2023, 8:46pm

I am able to reproduce the issue and have logged for investigation at `GeometryViewer` WebXR option does not appear on local deployment · Issue #2674 · Kitware/vtk-js · GitHub.

A workaround in the meantime is to use adb for wireless debugging at localhost:

Make sure your Android device is in developer mode
Connect your Android device to your PC
Open a console on your PC and run adb devices and verify your device is present
Run adb tcpip 5555 to listen for a connection on port 5555
Disconnect your device from the PC
Find the wireless IP address for your device under wi-fi settings
On your PC run adb connect <your-device-ip> to connect to the device over wi-fi
Run adb reverse tcp:9999 tcp:9999 to allow adb to route traffic between port 9999 on both the PC and mobile device
Build and run GeometryViewer on your PC and visit the localhost example address on your mobile device. For the diskout.vtp example this URL is http://localhost:9999/?fileURL=https://data.kitware.com/api/v1/item/59de9de58d777f31ac641dc5/download .

Lois_Vazquez · January 23, 2023, 10:09am

Thank you for such a quick response.

My mobile device is: Xiaomi Mi 11 (the same result with other devices)
Browsers in PC: Google Chrome and Mozilla Firefox
Browsers in Android: Google Chrome

Check results in PC with Google Chrome and Mozilla Firefox:
navigator.xr !== undefined: true
navigator.xr.isSessionSupported('immersive-ar'): [object Promise]
fullScreenRenderWindow.getApiSpecificRenderWindow().getXrSupported(): true

Check results in Android with Google Chrome:
navigator.xr !== undefined: false
navigator.xr.isSessionSupported('immersive-ar'): nothing is showed
fullScreenRenderWindow.getApiSpecificRenderWindow().getXrSupported(): true

In the past days I found that even with your example without HTTPS, WebXR support is not available on my mobile device. I guess it’s the same reason it happens on localhost.

Alejandro_De_Leon_Cu · March 6, 2023, 10:36pm

I have the same problem. I can’t get the Start AR button to appear. I try to access the sample data remotely but it’s the same. In the browser debugging tool m I get the following error:
Uncaught (in promise) Error: Failed to create XR session!
I have also tried with the basic example of vtk.js but there it does not execute any action when I press the button from my mobile.

tbirdso · March 7, 2023, 12:39am

Hi there, yes, this sounds like the same issue as above where WebXR is not supported on all mobile devices and web browsers. This is likely a limitation of your hardware or operating system.

You can consult the following resources to determine whether your mobile device supports WebXR:

WebXR supported browsers

For Android devices: ARCore supported devices

For iOS devices: ARKit supported devices

Alejandro_De_Leon_Cu · March 7, 2023, 1:03am

Thank you for the prompt response.
Too bad :(. But, does this apply if I don’t have this problem accessing the example sites at vtk.js and https://kitware.github. io/vtk-js/examples/GeometryViewer.html?
from that sites it works fine on the two devices I have, unlike if I try to access from the localhost

tbirdso · March 7, 2023, 2:12am

from that sites it works fine on the two devices I have, unlike if I try to access from the localhost

Okay, good to know, in that case it sounds like your device does support WebXR.

Are you unable to launch the AR view when the example is running on your PC and your mobile device is connected via USB to your PC?

Does the workaround described above improve your experience?

If you are using the Chrome WebXR emulator extension (or the Firefox equivalent), when you build the example and access at localhost are you then able to launch into AR as expected?

tbirdso · May 11, 2023, 5:01pm

Hi all,

Revisiting this thread, we’ve determined that the issue described above was likely due to serving examples over an unsecured HTTP context by default. WebXR requires a secure context to run (see the WebXR security explainer).

To resolve the issue we’ve added a script to run examples locally over HTTPS with a self-signed SSL certificate in 79d7927f. Use the following steps to run vtk.js WebXR examples from your local PC:

On the host PC, serve an example locally over HTTPS:

path/to/vtk-js> npm run example:https -- WebXRVolume
...
<i> [webpack-dev-server] Project is running at:
<i> [webpack-dev-server] Loopback: https://localhost:9999/
<i> [webpack-dev-server] On Your Network (IPv4): https://xxx.xxx.xxx.xxx:9999/
...

In a browser with WebXR support on the XR device, navigate to the IPv4 address given above, optionally adding URL arguments for the example

Please let us know here if you experience further issues in running vtk.js WebXR examples.