Multi-User Cross-Device Remote Rendering with Local Positioning Assistance for Mixed Reality Experiences

以作者查詢圖書館館藏

、以作者查詢臺灣博碩士

、以作者查詢全國書目

、勘誤回報

、線上人數：92

、訪客IP：3.17.6.75

姓名

黃正婷(Zheng-Ting Huang) 查詢紙本館藏

畢業系所

通訊工程學系

論文名稱

(Multi-User Cross-Device Remote Rendering with Local Positioning Assistance for Mixed Reality Experiences)

相關論文

★ 基於馬賽克特性之低失真實體電路佈局保密技術	★ 多路徑傳輸控制協定下從無線區域網路到行動網路之無縫換手
★ 感知網路下具預算限制之異質性子頻段分配	★ 下行服務品質排程在多天線傳輸環境下的效能評估
★ 多路徑傳輸控制協定下之整合型壅塞及路徑控制	★ Opportunistic Scheduling for Multicast over Wireless Networks
★ 適用多用戶多輸出輸入系統之低複雜度比例公平性排程設計	★ 利用混合式天線分配之 LTE 異質網路 UE 與 MIMO 模式選擇
★ 基於有限預算標價式拍賣之異質性頻譜分配方法	★ 適用於 MTC 裝置 ID 共享情境之排程式分群方法
★ Efficient Two-Way Vertical Handover with Multipath TCP	★ 多路徑傳輸控制協定下可亂序傳輸之壅塞及排程控制
★ 移動網路下適用於閘道重置之群體換手機制	★ 使用率能小型基地台之拍賣是行動數據分流方法
★ 高速鐵路環境下之通道預測暨比例公平性排程設計	★ 用於行動網路效能評估之混合式物聯網流量產生器

檔案

[Endnote RIS 格式]

[Bibtex 格式]

[相關文章]

[文章引用]

[完整記錄]

[館藏目錄]

至系統瀏覽論文 (2025-8-30以後開放)

摘要(中)

隨著科技快速的發展下，增強現實（AR）、虛擬現實（VR）和混合現實（MR）等技術成為一個備受關注的領域。然而這些技術在遠端算繪應用中仍然需要面臨一些困難和問題，例如延遲和預測不準確等。為了防止出現這些問題，本文提出了一種多人遠端算繪架構，目標於提供良好的MR應用體驗。該架構利用伺服器端物件串流相機設計和本地端定位輔助，幾乎消除了延遲帶來的不順暢物件位移現象。同時，通過最佳化網路流量，考慮「人與物件」之間的關係，透過動態調整解析度以及視野剔除或合併物件等方法，提高了網路使用效率和提升用戶體驗。實驗結果顯示，該架構的數據最佳化方法平均降低了80%的資料量，與傳統的MR串流架構相比減少了70%。此外，該架構還支持多個用戶使用不同廠牌的頭戴顯示設備或行動設備加入虛擬空間並進行物件互動，並減少了伺服器上的硬體消耗。

摘要(英)

In the wave of rapid development of modern technology, augmented reality (AR), virtual reality (VR), and mixed reality (MR) have become prominent fields. However, these technologies still face some challenges in remote rendering applications, such as latency and inaccurate prediction. To address these issues, this paper proposes a multi-user remote rendering architecture aimed at providing users with a high-quality MR application experience.
The architecture utilizes a server-side object streaming camera design and local positioning assistance to virtually eliminate the noticeable object displacement caused by latency. Simultaneously, by optimizing network traffic, considering the relationship between users and objects, dynamically adjusting resolutions, and performing view frustum culling or object merging, the network efficiency is improved, and the user experience is enhanced.
Experimental results demonstrate that the data optimization method of this architecture reduces the average data volume by 80%, compared to a traditional MR streaming architecture, which reduces the data volume by 70%. Additionally, this architecture supports multiple users using different brands of head-mounted displays or mobile devices to enter the virtual space and interact with objects, while reducing hardware consumption on the server.

關鍵字(中)

★ 擴增實境
★ 虛擬實境
★ 混合實境
★ 多人遠端渲染
★ 延遲補償
★ 傳輸資料最佳化

關鍵字(英)

★ Augmented Reality
★ Virtual Reality
★ Mixed Reality
★ Multi-user Remote Rendering
★ latency compensation
★ Data Optimization for Transmission

論文目次

1 Introduction 1
1.1 Background 1
1.2 Motivation 1
1.3 Contribution 2
1.4 Framework 3
2 Related Works 4
2.1 Remote Rendering System 4
2.2 Latency Compensation Method 5
3 MR Remote Rendering System and Position Error 7
4 Multi-user Localization-Assisted Remote Rendering 9
4.1 Mixed Reality WebRTC 10
4.2 Server Architecture 11
4.3 Client Architecture 12
4.4 Object Streaming Camera 12
5 Optimize streaming data volume 15
5.1 Frustum Culling 15
5.2 Resolution Adjustment 16
5.3 Merge 17
6 Experimental Results 21
6.1 Simulation Setup 21
6.2 Position Error Comparison 21
6.3 Transmission Data Volume Performance 23
6.4 Server Hardware Efficiency Performance 24
6.5 Optimizing data volume Performance 26
6.6 Actual Implementation Result 29
7 Conclusion and Future Work 30
7.1 Conclusion 30
7.2 Future Work 30
Bibliography 31

參考文獻

[1] S. Orts-Escolano, C. Rhemann, S. Fanello, W. Chang, A. Kowdle, Y. Degtyarev,
D. Kim, P. L. Davidson, S. Khamis, M. Dou et al., “Holoportation: Virtual 3d
teleportation in real-time,” in Proceedings of the 29th annual symposium on user
interface software and technology, 2016, pp. 741–754.
[2] S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, P. A. Chou, R. A. Co-
hen, M. Krivoku ́ca, S. Lasserre, Z. Li et al., “Emerging mpeg standards for point
cloud compression,” IEEE Journal on Emerging and Selected Topics in Circuits and
Systems, vol. 9, no. 1, pp. 133–148, 2018.
[3] L. Song, A. Chen, Z. Li, Z. Chen, L. Chen, J. Yuan, Y. Xu, and A. Geiger, “Nerf-
player: A streamable dynamic scene representation with decomposed neural radi-
ance fields,” IEEE Transactions on Visualization and Computer Graphics, vol. 29,
no. 5, pp. 2732–2742, 2023.
[4] S. Shi and C.-H. Hsu, “A survey of interactive remote rendering systems,” ACM
Computing Surveys (CSUR), vol. 47, no. 4, pp. 1–29, 2015.
[5] S. G ̈ul, D. Podborski, J. Son, G. S. Bhullar, T. Buchholz, T. Schierl, and C. Hellge,
“Cloud rendering-based volumetric video streaming system for mixed reality ser-
vices,” in Proceedings of the 11th ACM multimedia systems conference, 2020, pp.
357–360.
[6] J. Zhao, R. S. Allison, M. Vinnikov, and S. Jennings, “Estimating the motion-
to-photon latency in head mounted displays,” in 2017 IEEE Virtual Reality (VR).
IEEE, 2017, pp. 313–314.
[7] R. T. Azuma, “Predictive tracking for augmented reality,” Ph.D. dissertation, Uni-
versity of North Carolina at Chapel Hill, 1995.
[8] M. A. Livingston and Z. Ai, “The effect of registration error on tracking distant
augmented objects,” in 2008 7th IEEE/ACM International Symposium on Mixed and
Augmented Reality. IEEE, 2008, pp. 77–86.
[9] S. G ̈ul, S. Bosse, D. Podborski, T. Schierl, and C. Hellge, “Kalman filter-based head
motion prediction for cloud-based mixed reality,” in Proceedings of the 28th ACM
International Conference on Multimedia, 2020, pp. 3632–3641.
[10] A. Alahi, K. Goel, V. Ramanathan, A. Robicquet, L. Fei-Fei, and S. Savarese, “So-
cial lstm: Human trajectory prediction in crowded spaces,” in Proceedings of the
IEEE conference on computer vision and pattern recognition, 2016, pp. 961–971.
[11] X. Hou and S. Dey, “Motion prediction and pre-rendering at the edge to enable ultra-
low latency mobile 6dof experiences,” IEEE Open Journal of the Communications
Society, vol. 1, pp. 1674–1690, 2020.
[12] S.-P. Chuah, C. Yuen, and N.-M. Cheung, “Cloud gaming: a green solution to mas-
sive multiplayer online games,” IEEE Wireless Communications, vol. 21, no. 4, pp.
78–87, 2014.
[13] T. K ̈am ̈ar ̈ainen, M. Siekkinen, J. Eerik ̈ainen, and A. Yl ̈a-J ̈a ̈aski, “Cloudvr: Cloud
accelerated interactive mobile virtual reality,” in Proceedings of the 26th ACM inter-
national conference on Multimedia, 2018, pp. 1181–1189.
[14] Z. Long, H. Dong, and A. El Saddik, “Interacting with new york city data by hololens
through remote rendering,” IEEE Consumer Electronics Magazine, vol. 11, no. 5, pp.
64–72, 2022.
[15] S. Shi, K. Nahrstedt, and R. Campbell, “A real-time remote rendering system for
interactive mobile graphics,” ACM Transactions on Multimedia Computing, Com-
munications, and Applications (TOMM), vol. 8, no. 3s, pp. 1–20, 2012.
[16] D. C. Barboza, D. C. Muchaluat-Saade, E. W. G. Clua, and D. G. Passos, “An ar-
chitecture for multi-layer object coding in 2d game streaming using shared data in a
multi-user environment,” Entertainment Computing, vol. 42, p. 100490, 2022.
[17] S. Liu, X. Xu, and M. Claypool, “A survey and taxonomy of latency compensation
techniques for network computer games,” ACM Computing Surveys (CSUR), 2022.
[18] S. G ̈ul, D. Podborski, T. Buchholz, T. Schierl, and C. Hellge, “Low-latency cloud-
based volumetric video streaming using head motion prediction,” in Proceedings of
the 30th ACM Workshop on Network and Operating Systems Support for Digital
Audio and Video, 2020, pp. 27–33.
[19] S. Yoon, H. jeong Lim, J. H. Kim, H.-S. Lee, Y.-T. Kim, and S. Sull, “Deep 6-dof
head motion prediction for latency in lightweight augmented reality glasses,” in 2022
IEEE International Conference on Consumer Electronics (ICCE). IEEE, 2022, pp.
1–6.
[20] H.-L. Zeng, “Larr:delay compensation via local positioning for mr remote render-
ing,” 6 2022.
[21] M. Ester, H.-P. Kriegel, J. Sander, X. Xu et al., “A density-based algorithm for dis-
covering clusters in large spatial databases with noise.” in kdd, vol. 96, no. 34, 1996,
pp. 226–231.
[22] “4dviews volumetric motion capture systems.” https://www.4dviews.com/
volumetric-resources.
[23] S. G ̈ul, S. Bosse, D. Podborski, T. Schierl, C. Hellge, M. A. Kastner, and J. Zah ́alka,
“Reproducibility companion paper: Kalman filter-based head motion prediction for
cloud-based mixed reality,” in Proceedings of the 29th ACM International Confer-
ence on Multimedia, 2021, pp. 3619–3621.

指導教授

黃志煒(Chih-Wei Huang)

審核日期

2023-8-15

推文