👨💻 Building Open Source Showcases and Samples @RealityLabs 🐋 Teaching XR on YT with XR Dev Rob🇨🇭 🥽 github.com/xrdevrob/QuestCam…
Zurich, Switzerland
Joined May 2021
- Tweets 1,241
- Following 365
- Followers 1,449
- Likes 2,010
We have been working hard on something special for you guys - Access to the Meta Quest cameras👀
👨🏫 Learn all about it here: invidious.tiekoetter.com/1z3pcMJbnRA
⭐️ Check out my samples here: github.com/xrdevrob/QuestCam…
Our official Passthrough Camera Samples: github.com/oculus-samples/Un…
Documentation: developers.meta.com/horizon/…
24
59
31
282
0
🚨 QuestCameraKit has a new contributor!
Massive thanks to @devtom7 for contributing his shader experiments to the repo.
If you'd like to contribute to the QuestCameraKit, we are still looking to cover other use cases such as stable diffusion, frosted glass shader, image to 3D, and more!
3
11
1
97
0
QuestCameraKit has its first contributor!🙌
FireDragonGameStudio contributed his WebRTC + WebSocket setup to stream the Passthrough Camera from the Quest to other clients and various devices.
This is huge for anyone working on remote assistance, web calling or other collaborative use cases!
Go show invidious.tiekoetter.com/@firedragongames… some love!
If you'd like to contribute your own unique use case that explores the use of Passthrough Camera on Quest, check out github.com/xrdevrob/QuestCam…!
12
2
100
GIF
Devs who are already working with Passthrough Camera API mentioned above: @uralstechCTO @devtom7 @Taka_Yoshinaga @lirezaBahremand @thorikawa @smallfly @mizzmayo @senooyudai @Dilmerv @SkarredGhost
2
12
I just added a new "Community Contributions" section to github.com/xrdevrob/QuestCam…, where you can find interesting prototypes that use the Passthrough Camera API or were built with QuestCameraKit.
Feel free to link any cool demos I have forgotten in the comments below! 👇
6
7
1
60
Robi ᯅ retweeted
自分が公開していたXREAL用のMRTKプラグインの解説動画を作ってくれたの嬉しかったなあ。そんな彼が作ったQuestのPassthrough Camera APIのサンプルで今日は勉強させてもらってよても良いご縁を感じる。一昨年福岡であったのも懐かしい。
Creating things in public/open source is the one and only Virtuous Cycle.
3 years ago, I made my first ever YouTube tutorial (please don't watch it) about hand tracking with Nreal light (now @XREAL_Global) using @Taka_Yoshinaga's GitHub project github.com/TakashiYoshinaga/…
Later, I had the pleasure to meet Takashi in Fukuoka in October of 2023, just when the Quest 3 was released (see images below).
Today, I am happy to see inspiring figures like Takashi himself use my GitHub samples to get started with Passthrough Camera access on Meta Quest github.com/oculus-samples/Un… and github.com/xrdevrob/QuestCam…
nitter.tiekoetter.com/Tks_Yoshinaga/status/1…
I wouldn't be here without these devs and I am super grateful to finally be able to give something back.
5
39
Creating things in public/open source is the one and only Virtuous Cycle.
3 years ago, I made my first ever YouTube tutorial (please don't watch it) about hand tracking with Nreal light (now @XREAL_Global) using @Taka_Yoshinaga's GitHub project github.com/TakashiYoshinaga/…
Later, I had the pleasure to meet Takashi in Fukuoka in October of 2023, just when the Quest 3 was released (see images below).
Today, I am happy to see inspiring figures like Takashi himself use my GitHub samples to get started with Passthrough Camera access on Meta Quest github.com/oculus-samples/Un… and github.com/xrdevrob/QuestCam…
nitter.tiekoetter.com/Tks_Yoshinaga/status/1…
I wouldn't be here without these devs and I am super grateful to finally be able to give something back.
5
6
2
60
Robi ᯅ retweeted
鏡に写る自分を見たらずっと美少女なんじゃ？！と思って洗面台に行ったらずっと僕が美少女でした。HMD被ったままだけどSDXLのImg2Img過程で無視されてくれて助かった。もうずっとQuest3付けたまま生活したい🤤
2
21
1
46
0
Robi ᯅ retweeted
As I've said before: this is one of the most significant Quest SDK updates ever, dramatically expanding what developers can build.
Quest's highly anticipated Passthrough Camera API is now available for all developers to experiment with, though they can't yet include it in store app builds:
uploadvr.com/quest-passthrou…
6
13
121
Passthrough Camera access (PCA) on Quest may have taken a while and I already see some devs doubting the impact this will have for XR.
I think the most fun thing about PCA is, that there are such smart devs out there, that we cannot even begin to imagine what some people will come up with, and I am very much looking forward to that.
Below is another sample I made and included in my QuestCameraKit GitHub repo. It is using API calls to OpenAI vision and whisper and is capable of understanding and responding in virtually any language (English and German below): github.com/xrdevrob/QuestCam…
5
21
1
176
0
Quest's highly anticipated Passthrough Camera API is now available for all developers to experiment with, though they can't yet include it in store app builds:
uploadvr.com/quest-passthrou…
38
7
188
Robi ᯅ retweeted
Colocated mixed reality experiences can fuel engagement and retention by supporting greater social presence in a shared environment. 👥 Watch the tutorial below from @Dilmerv for steps on using multiplayer Building Blocks to create a colocated game👇
3
13
76
I want to give a huge shout out to all the passionate devs who have been pushing for Camera access, have put in the work and have even enabled other developer to use workarounds to build out first prototypes.
@trev3d: github.com/trev3d/QuestDispl…
@lukas_moro: nitter.tiekoetter.com/lukas_moro/status/1794…
@lucas_martinic: nitter.tiekoetter.com/lucas_martinic/status/…
@mechpil0t: nitter.tiekoetter.com/mechpil0t/status/18568…
and many more.
Inspired by this exploration from @ConcreteSciFi I built an implementation of object detection on the pass-through for a Unity application on Quest 3 using YOLOv8.
2
3
28
@iBrews @mechpil0t @cixliv @Heaney555 @jmdagdelen @augmentedcamel @aurepuerta_dev @ID_AA_Carmack @marknb00 @XRarchitect @SpatialBiggs @InnerBushido @Taka_Yoshinaga @ValemVR @Aidan_Wolf @GabRoXR @devtom7 @lucas_martinic @_LucasRizzotto @trev3d @MatthewHallberg @SkarredGhost @stspanho @RaelXRWorkout @realdowd @Immersal_AR @r0bwiki
5
15
Robi ᯅ retweeted
If you are thinking about VR/MR development with Meta SDKs, I recommend looking at this amazing resource by @xrdevrob which shows you some of the best practices when developing common features that most devs may need for games or apps.
📌 Current Motifs (best practices/patterns)
- Passthrough Transitioning - Seamlessly fade between Passthrough and VR
- Shared Activities in Mixed Reality - Make people feel truly physically present with each other
- Instant Content Placement - Use the Depth API to create effects not possible otherwise
💡GitHub repo link 👇
3
8
79
👨💻 Working on a Meta Quest demo using OpenAI's whisper and vision models, and I only spent $0.20 during the whole development (69 requests & 18.758K input tokens).
I am wondering, are there already any Quest apps out there letting the user enter their own API key? Or how else is pricing usually handled?
3
11
Robi ᯅ retweeted
Enable seamless content placement with the new Instant Content Placement Mixed Reality Motif.✨ This project illustrates how to use the Depth API and Raycast API to unlock realistic interactions, lighting and layering between virtual and physical objects.💫 Get started below.👇
2
10
1
65