How To Fly Your DJI Drone With Apple Vision Pro

Okay, so I’ve spent the past couple of days flying my drones with Apple’s Vision Pro headset. While the experience has been great—I think it actually makes my time flying more enjoyable—the process is fairly complicated to get these to actually work. It doesn’t just bind and fly. You can’t simply connect it to your phone or your drone and then just go. You’ve got to go through this whole entire process, which makes it better for those times where you go somewhere and are going to be flying for, say, a couple of hours rather than going out and flying one quick battery to grab a couple of quick photographs.

So, what I want to do first is show you my setup when I fly with Vision Pro, like inside of my goggles, and all of the apps that I use. And then, I’ll show you how to actually get the live feed from your phone when flying your drone to your goggles. Because, in typical Apple fashion, they certainly don’t make it easy.

So, remember, Apple Vision Pro gives you the ability to be fully immersed inside of the goggles. But when it comes to flying, it’s just best to turn that immersion off and use Vision Pro as a set of augmented reality goggles instead of virtual reality. Like when I’m out flying my drones, taking photos and videos, I want to have complete situational awareness.

I want to know what’s going on around me. I want to be able to interact with a client and look at them directly. I want to be able to talk with friends that I might be flying with. I just want to know what’s going on around me and not be cut off from the outside world. So, that’s why I prefer an augmented reality headset in this regard.

I would, however, prefer to be cut off inside of a pair of goggles when I’m flying FPV. So, like when I take my Avada out, I prefer to use DJI’s own goggles, the Goggles 2 or the Goggles Integra, so that I get that fully immersive experience when I’m ripping around using that DJI Mavic 3 drone.

Now, getting back on track here with Vision Pro, because you aren’t fully immersed, the real reason for using goggles like this is to cut out the sun glare on your screen, protect your eyes from squinting like crazy, it gives you a bigger screen to look at, and it lets you comfortably sit with your head elevated as opposed to slouching over looking at your remote.

This leads to an overall much better flying experience. It allows me to frame my shots better, gives me a better look at the actual framing of the photos and videos that I’m taking. It lets me properly set my exposure without having to try and beat the sun glare and slouch over and protect the screen from the sun using my back.

And I think that, when we get into the summertime, being able to avoid that sun glare is just going to be so key, especially on those hot afternoons where the sun is high in the sky, beating down. Being able to throw these on and comfortably see my screen is going to be a game-changer.

Okay, so now let me bring you into my Vision Pro to show you how I set things up. I typically will sit in the trunk of my car so I’m comfortable, and then I look out to have a big canvas that I can store all of my windows. So, obviously, right in the middle, I have the live feed coming from my drone, which is the most important thing. So, that is big and front and center.

From here, I’ll usually have Apple Music open in some area so that I can play some music in the background. I love flying to music because it helps me get in the zone. So, being able to play it low without anyone else hearing is nice.

Now, depending on the job I’m on, I might also have a set of plans or a note with a shot list or some type of reference pulled up so that I know what I need to do and what I need to capture. This is something that I used to do by trying to glance at my phone, but now being able to just look over at another window makes it so much more efficient to direct my attention between both things.

The final thing I like to do is bring up a Safari window and sit it just next to me, that I’ll have to look at any sort of information, whether it’s map info, weather info, or maybe it’s even a YouTube video that I’m playing in the background instead of listening to music. Regardless, there are plenty of different applications that you can have open with Vision Pro.

There’s a lot of apps built into the goggles already, and there are more coming each and every day. So, being able to display all these apps in one area and just glance around as you fly really does make things easy.

Like, something else that I found really powerful is being able to view photos or videos to reference what you might be shooting. Like, for example, if I’m photographing the skyline and have some old shots that I want to replicate, I could directly review a photo or video clip and then try to match it with my flight screen side by side.

I’ve also been doing some of these day-to-night transitions for some of the videos that I’ve been creating of homes here in Philadelphia. So, being able to directly compare my daytime shot to my nighttime shot, or vice versa, really makes getting that transition super easy.

So, when I first picked up the Vision Pro, I didn’t think they’d be such a game changer for me, but they really have changed the way that I fly. Like, being able to have all the different windows open makes this a multitasker’s dream. But really, I just like the fact that I can cut out all the sunlight and get a clear view of my flight screen. It’s a pretty solid connection too, with a little bit more lag than I’m used to, but hopefully, as Apple Vision Pro receives updates, it’ll become a more seamless experience.

How to connect your iPhone or iPad and Apple Vision Pro

And with that, it now brings us to the second part of the video, where we discuss how to get your iPhone or iPad screen displayed on your goggles. Because, as I mentioned, it is not as seamless of an experience as you’d think.

So, you’d think that because Vision Pro is within Apple’s ecosystem, it would be easy to cast whatever’s on your iPhone or iPad screen directly onto your goggles, but that is not the case, as Apple hasn’t built this feature into these goggles yet. I do want to mention that Vision Pro can basically run any app on the App Store out in space, as if it’s on an iPhone or an iPad. So, for example, the DJI Fly App is already on the Vision Pro and boots up just like any other application.

But since you can’t plug your remote into the goggles, you can’t interface with the app directly. This means, in order to make this work, we need to take advantage of the virtual Mac desktop feature on Apple Vision Pro so that we can see what’s on our Mac screen as a movable window inside of Vision Pro.

So, this right here is everything you need. Of course, you need your drone, which in this case I’ve got my Mavic 3 Pro. Of course, you need Vision Pro, but in order to make this work, you also need a controller that allows you to interface with your iPhone, so you can’t be using the Smart Controller, the RC Pro, the RC, or the RC2 from DJI.

You need something that allows you to connect to your phone and then display the Fly app on your iPhone or your iPad. With that said, you also, of course, need an iPhone, you also need a MacBook, and then you need an application called Reflector running on your MacBook.

And finally, you need a dedicated Wi-Fi connection. So, this means you can’t be using your personal hotspot from your iPhone that’s used to fly your drone. You’re going to need something different. You need a personal hotspot, you need another iPad with an LTE connection, you might be able to use a Wi-Fi connection directly on-site, but regardless of what the connection is, it can’t be your phone.

Now, I know that this all sounds quite cumbersome. I mean, this list right here is everything you need just to make this setup work. And this process will probably be way different in a few months from now, but for now, this is very much a workaround because Apple hasn’t given us the ability to natively display our iPhone screen inside of the Vision Pro, and the DJI Fly app just isn’t optimized either. Either of those fixes would make this process way easier.

Okay, so I know that I probably look pretty ridiculous wearing these goggles. I’m pretty sure that you can see my eyes through the screen, which just makes these very weird. But regardless, we’ve got everything in place to get this setup. So, we’ve got the drone turned on, we’ve got our remote controller turned on with the Fly app open on our iPhone, we’ve got our MacBook turned on with Reflector running in the background, and of course, we have the Vision Pro turned on.

So, from here, the first thing that we need to do is get our phone screen displayed on the computer. So, really, from here, we just need to swipe down to the control center, we need to swipe over to screen mirroring, and then we’ll go and select William’s MacBook Pro. Now, notice it actually has an Apple TV logo next to it, and that’s because we are running Reflector here on the computer, kind of making this look like an Apple TV, but not a computer.

Also, you want to make sure on your computer you’ve got AirPlay Receiver turned off, so that it just looks like your MacBook Pro here being the Apple TV logo.

So, we’ll tap on that, it now shows us the screen from our iPhone on the computer, and then from here, we’ll go and tap on Connect, and now connects Vision Pro to the Mac virtual desktop, and now, voila, we have our phone displaying directly on our Vision Pro.

From here, we can go and tap back, we now have a full visual of the actual screen here from the controller. So, if we go and look up and down, it’s going to show us the screen, which is great. We can fly first-person view from here.

Now, using our MacBook, we can actually make this full screen by going to the top, the Reflector icon, and then making it full screen. And now, we are just looking at the screen of our drone, which still is very impressive.

So, again, a little bit annoying trying to set all this up. It’s funny because right now, I can’t see the camera, but I know that you guys can still see me because of the virtual window. But this is great, like, I know that I can take this and make this huge. Like, this screen is now massive in front of me. As I showed in my first flight video, I can actually make this the size of my car. I can resize this and move it around. So, this setup here, if I could get it to work, really is awesome.

I’m going to go ahead and close this. I’m going to go ahead and take off Vision Pro. Actually, I’m going to stop my screen recording here first. Here, stop. So, this whole entire setup here really is great. I mean, being able to have all the windows around you is awesome.

I’ll give you guys a little glimpse into my setup. Usually, my trunk is fairly clean. I don’t keep anything unnecessary back there, so I shove my backpack in the corner, I shove my MacBook in the corner. I’ll also have a little battery bank that charges the battery of the Apple Vision Pro as well as my MacBook, so I’m always topped up there. And then, from there, I just fly. I sit in the back of my seat, or the back of my trunk, I look at the Apple Vision Pro screen through my goggles, and fly, and it really does make all the difference.

So, yes, I know that this is definitely quite cumbersome right now, but for me, it’s worth it. I’ve done it now a couple of times while I’m out flying, and it actually is easy to get set up. And once you have everything running, you can kind of put it inside of your car and then forget about it. And just keep using your Vision Pro and flying your drone like you normally would. And then, you just have to close your computer when you’re all done.

The biggest key is to make sure you have that dedicated hotspot or that dedicated Wi-Fi connection. So, again, in my apartment, I’ve got that Wi-Fi connection. I don’t really care about having to use a personal hotspot, but when you go through this process, casting your phone to your computer, you can’t be using the personal hotspot, as it disables Wi-Fi.

So, having some sort of dedicated connection, for me, I have like a personal hotspot from Verizon. I don’t know exactly which one it is, but it gives me a nice 5G connection, so I can connect my computer, my phone, as well as my goggles to that Wi-Fi connection and have internet wherever I go. So, investing in something like that could definitely be very important for this setup to work.

Anyway, thank you guys so much for watching. I’ll definitely keep you in the loop as time progresses. If DJI comes out with a native version of the Fly app for the Apple Vision Pro, or if Apple makes this process easier by allowing us to cast the iPhone to the goggles, nonetheless, I’ll be sure to keep you guys updated. So, stay tuned to the channel, and as always, I’ll talk to you later. Peace.