- 166
- 1 121 304
Matthew Brennan
United States
Приєднався 5 лют 2010
Hi, I'm Matthew Brennan, a PhD student focusing on (immersive) technologies applied to art, architecture, archaeology, and cultural heritage.
Total Solar Eclipse 2024 360 VR
360 video of the April 8, 2024 total solar eclipse; Bloomington, IN.
100% totality starts around 1:00 but I recommend watching from at least 0:50 onward to get a sense of the change in light and mood as the eclipse happens!
100% totality starts around 1:00 but I recommend watching from at least 0:50 onward to get a sense of the change in light and mood as the eclipse happens!
Переглядів: 302
Відео
360VR Photogrammetry while exploring Hoosier National Forest
Переглядів 1912 місяці тому
Fixed the stitching errors at nadir and zenith :) Hickory Grove Church Gaussian Splat cloud on PolyCam: poly.cam/capture/8a0a7f84-c955-4b76-bb6e-e302f111dd09 Photogrammetry mesh model on Sketchfab: skfb.ly/oSOFY Filmed on an Insta360 OneR. Chapters: 0:57 Intro and driving to Hoosier National Forest 06:40 Pleasant Run Church 08:00 Pleasant Run Cemetery Walkaround 13:19 Hickory Grove Church 20:25...
Indiana University Greenhouse 360VR
Переглядів 2052 місяці тому
360 VR 8K spherical captures of the Indiana University Greenhouse. Spherical images captured with a Sony A7Rii with a Sony 12-24mm G lens on a nodal tripod head. 23-25 images per panorama processed in Hugin.
33mins of puppies playing and napping in 360 degrees
Переглядів 753 місяці тому
360 video of 6 four-week-old foster puppies playing and napping in their enclosure.
Piazza Navona: 3D Gaussian Splatting and Photogrammetry
Переглядів 7114 місяці тому
Comparison and discussion of 3D Gaussian Splat visualization vs. Photogrammetry for a complex sculptural object - The Fountain of the Four Rivers in Piazza Navona, Rome.
Moab - Denver 360 Hyperlapse at Sunrise
Переглядів 954 місяці тому
360-degree hyperlapse of a sunrise drive from Moab, Utah to Denver, Colorado (~350 miles, ~560 km, or ~5hrs). Filmed in timelapse mode with an Insta360 One R, mounted to the rear window of a Toyota Tacoma TRD OR. Chapters: 0:41 Arches National Park (you can see taillights going up the hill on right) 2:35 Sunrise on I-70 10:12 Glenwood Springs / Aspen
Photogrammetry Turntable Workflow - Agisoft Metashape and ZBrush
Переглядів 3744 місяці тому
"Turntable Workflow" walkthrough for object digitization using Agisoft Metashape. Link to the final model on Sketchfab.com: skfb.ly/oQynp This is a long one, so buckle up and grab a coffee. Chapters: 0:00 Intro 2:30 Workflow overview 9:20 Turntable Photography 20:50 Handheld Photography Workflow 27:15 Processing the Images 30:50 Aligning images in Metashape 35:50 Generating Masks from Model 45:...
3D Gaussian Splatting Hadrian's Villa
Переглядів 1,5 тис.4 місяці тому
Some 3D Gaussian Splat visualizations of Hadrian's Villa compared to photogrammetry. Photogrammetry model of the Plastico on Sketchfab: skfb.ly/oxxMF Digital Hadrian's Villa Project video: ua-cam.com/video/zGdjf9wzHOI/v-deo.html
10 hrs in 5 mins; Indiana to Maryland Space-Time Compression
Переглядів 1285 місяців тому
Ten and a half hour drivelapse. 1 photo every 2 seconds. GoPro Hero4 with lens modification. Starting before dawn in Bloomington, Indiana, ending after dark in Baltimore, Maryland. Music and Sound credits: Back to Lighted Streets by Kai Engel; freemusicarchive.org rain_windshield_wipers_sanus_excipio.wav; freesound.org
3D Gaussian Splatting the Medici Tombs
Переглядів 1,9 тис.6 місяців тому
3D Gaussian Splat rendering of the Medici Tombs photogrammetry dataset. The Medici Tombs are sculptures by Michelangelo Buonarroti, located in the New Sacristy of the Basilica San Lorenzo in Florence, Italy. Arena Chapel 360* video rendered completely from a photogrammetry dataset: ua-cam.com/video/QVUVVQ5doZs/v-deo.html
Gaussian Splatting Large Urban Spaces vs. Photogrammetry
Переглядів 6 тис.7 місяців тому
3D Gaussian Splatting for visualizing large unbounded scenes. This isn't really the intended application of 3DGS (per the authors, here: github.com/graphdeco-inria/gaussian-splatting#faq) but with some modified settings, it works quite well for large urban scenes.
VR in the Art History Classroom: From Concept to Implementation
Переглядів 8297 місяців тому
This is a quick overview of my research and teaching using VR for art history education, beginning with the initial conception of the project in 2016 and culminating with the use of the VR environments in classrooms in 2021-22. Smarthistory Arena Chapel 360 Video: ua-cam.com/video/QVUVVQ5doZs/v-deo.htmlsi=MP4YeOpSqotqwEiE
Neural Surface Reconstruction (NeuS) vs. Photogrammetry
Переглядів 2,2 тис.7 місяців тому
Neuralangelo: research.nvidia.com/labs/dir/neuralangelo/ Kiri Engine: www.kiriengine.app/web-version/ Android: play.google.com/store/apps/details?id=com.kiriengine.app iOS: apps.apple.com/us/app/kiri-engine/id1577127142 Chimera of Arezzo photogrammetry model (note: LOTS of manual cleanup, back in 2016 or so): skfb.ly/OSr8 Chimera of Arezzo NeuS model: www.kiriengine.app/share/ShareModel?code=F7...
3D Gaussian Splatting Complex Spaces
Переглядів 2,4 тис.8 місяців тому
A compilation of 3D Gaussian splatted architectural interiors and urban spaces. Created for CAAV 2023. The scenes (and their architects/artists, when known): Pantheon, Rome Sant'Ivo alla Sapienza (Francesco Borromini), Rome Saint Teresa in Ecstasy (Gian Lorenzo Bernini), Santa Maria della Vittoria, Rome Sant'Andrea al Quirinale (Bernini), Rome San Carlo alle Quattro Fontane (Borromini), Rome Sp...
Photogrammetry Mesh vs. Gaussian Splatting point cloud
Переглядів 3,5 тис.8 місяців тому
Comparison of the same data set processed with photogrammetry and Gaussian splatting. Data: 110 images, 42mpx Photogrammetry: 6million polygons, 10 4096 textures Gaussian Splatting: 30k iterations, 172k points, 7 million splats
Gaussian Splatting San Carlo alle Quattro Fontane
Переглядів 6068 місяців тому
Gaussian Splatting San Carlo alle Quattro Fontane
Reflective Object: Gaussian Splatting radiance field vs. Photogrammetry mesh
Переглядів 2,5 тис.8 місяців тому
Reflective Object: Gaussian Splatting radiance field vs. Photogrammetry mesh
Gaussian Splatting and Photogrammetry with 360/spherical imagery
Переглядів 13 тис.8 місяців тому
Gaussian Splatting and Photogrammetry with 360/spherical imagery
Vernazza photogrammetry massing and light study
Переглядів 1,2 тис.8 місяців тому
Vernazza photogrammetry massing and light study
Photogrammetry / NeRF / Gaussian Splatting comparison
Переглядів 150 тис.8 місяців тому
Photogrammetry / NeRF / Gaussian Splatting comparison
Neural Radiance Renaissance - NeRF 'ing Renaissance Art and Architecture
Переглядів 2879 місяців тому
Neural Radiance Renaissance - NeRF 'ing Renaissance Art and Architecture
IU Metz Carillon Neural Radiance Field
Переглядів 2419 місяців тому
IU Metz Carillon Neural Radiance Field
Sunrise at Land Between the Lakes, April 2023
Переглядів 77Рік тому
Sunrise at Land Between the Lakes, April 2023
360-degree drive from Paynetown SRA to IU Bloomington, IN / 4K 360VR
Переглядів 201Рік тому
360-degree drive from Paynetown SRA to IU Bloomington, IN / 4K 360VR
Archaeological Analysis and Re-Presentation in Virtual Reality
Переглядів 215Рік тому
Archaeological Analysis and Re-Presentation in Virtual Reality
Reading Architecture in Virtual Reality: I.M. Pei's Library Plaza in Columbus, Indiana ( 8k 360 VR )
Переглядів 141Рік тому
Reading Architecture in Virtual Reality: I.M. Pei's Library Plaza in Columbus, Indiana ( 8k 360 VR )
I'm curious if you have used your Insta 360 1 for the photogrammetry? I know that Meshroom supports 360 images. There may be instances where you are unable to use the drone for aerial shots, but could mount your 360 camera on a long selfi-stick to get over the subject.
Thanks for posting this. Adding Houdini to your tool set will give you the ability to leverage volumes and point clouds to their fullest potential.
Enjoyable and educational! Thanks Matthew!
That was good man - thanks
Wonderfully well done! Keep it up: you'll answer a lot of unasked questions!
Immediately recognized Porto Venere! Absolutely unique place.
I'm not into software war camps, but why using zbrush for cleaning up if you can use blender for free? Is there a huge benefit sticking to zbrush?
I have used blender, I'm just more familiar with and more proficient with ZBrush. I also hate blender's interface :) It's one of the worst softwares I've ever used. However, it IS free. I last used it 3-4 years ago, maybe it has gotten better.
Very nice!
how did you get the camera positions?
photogrammetric alignment using Agisoft Metashape.
@@MatthewBrennan thank you!
I couldn’t find in the description and credits at the end of the video whose music was used here. Could you tell?
Sorry: "Elgar - Serenade for Strings", added to the video description too!
@@MatthewBrennan thank you very much
why not taking only keyframes from the pictures?
That’s cheating.
11:26 bro uploaded the video in 4k but recorded the screen with an amazon doorbell camera 💀
😂 yep. Lesson learned. The renderings were 4K but my screen recording was 1080 🥲
If only these beautiful buildings existed today, it would have been amazing. It’s an unbelievable engineering. The talents and skills of the builders were astounding.
Of course we have to remember, that slaves were worked very hard,even to death for these beautiful palaces.
Amazing use case!
Wow this is incredible work. Thank you.
Glad you liked it!
What type of 360 camera did you use?
Insta360 One R - I've had it for about 4 years and have mixed feelings about it - sometimes it's great, other times it will freeze and shut down for no reason - and to my eye the footage is decent. For the price it's a good little 360 camera.
what are your computer spec?
The 3DGS was trained on a cloud workstation instance with an A100 GPU. The video rendering and photogrammetry was done on a desktop with a Ryzen 9 3900X, 4070ti, and 32GB RAM.
Ok. Why did the psychedelic sky of the gaussian splat give me a panic attack? I'm going to call it a Splat Attack.
What music?
"Elgar - Serenade for Strings"
What is your drone?
Mavic 2
Very informative video. You are showing a textured model from photogrammetry in the end. How does one create that?
You need a photogrammetry software. In this case, I used Agisoft Metashape, but there are free/open-source options, such as COLMAP and VisualSFM
@@MatthewBrennan Thank you very much for that information. Much appreciated.
Thanks for the good video and for providing the nice dataset. What is the focal length of your drone's camera (full frame equivalent)?
35mm equivalent is ~28mm. It's a 1" CMOS 20mpx sensor (for still images). However this dataset uses 4k video.
@@MatthewBrennan thanks for your answer. I'm trying to run your dataset using micmac photogrammetry. I'll let you know if I get good results!
Cool.....but where's the moon?
During totality the moon moves in front of the sun (it's the black "hole" with sunlight shining around it) - when it's not occluding the sun, it's not visible (because the light is coming from behind it)... Unfortunately, the exposure settings on the 360 camera don't make it possible to really see the moon occluding the sun because there is enough light to still overexpose that area - but the shift from full/partial sunlight to eclipse is still pretty dramatic.
THANK YOU SOOO MUCH FOR THIS
Wow!!!
Someone should turn Google Street View into 3d. It would be so nice to drive around and explore the world in a driving game.
Is the photogrammetry done from VR footage?
No - the photogrammetry is a DJI mavic 2 and a Sony A7Rii. Although, as I was watching the 360 footage, I think I will try to digitize one of the grave markers using the spherical video (the one that I walk around completely).
Good old nodal stitch method, stunning results... Nothing beats that still 👏🏻👌🏻
Thank you! I was worried about ghosting on the greenhouse structure/mullions, but I must have had it dialed in perfectly :)
Haha, you definitely went on a challenge on that one, don't worry I've turned the stuff upside down all the way around... Flawless 😉 Sound atmosphere is also very immersive... I'm about to scan a very similar one... I'm concerned about the hours of deleting reflected points I'm gonna have to spend 😅
god damn insane
At about six minutes and thirty-nine seconds, what software was used?Looking forward to your reply😊
Agisoft Metashape - it's an industry-standard photogrammetry software (formerly called Photoscan).
That's an amazingly beautiful rock formation, not that it's the topic of the video but just felt like mentioning it
It has an interesting backstory too! The legend goes that a utopian community wanted to hollow out the rock to use as a church - and even went so far as to begin chiseling a doorway (you can see the opening in the video/model).
do you still recommend this technique ? I just saw this video that uses a slightly different technique and I'm wondering which one to use: ua-cam.com/video/LQNBTvgljAw/v-deo.html&ab_channel=TheNeRFGuru
360 imagery won't give you results like properly captured "photogrammetry"-style data - the typical 360 sensors are small, and you're cramming a lot of data into not-many-megapixels. If you absolutely have to use 360 data, good even scene lighting and a good 360 sensor are key, more than the processing technique.
Dude, your video looks horrible, why do you upload that like this with 144p resolution being in "HD" or "4K"?
Hi everyone, is it possible to digitize a vehicle so that it is displayed razor sharp with all of its flaws and scratches without using spray color?
You'd want to use LiDAR or structured light, or a combination of LiDAR and photogrammetry. I have digitized wrecked cars before using photogrammetry - and LiDAR is often used to document crash scenes.
Hi Matthew. Great video explanation. Which drone did you use for this test? Do you have by any chance any more drone footage? Have you use the DJI Mavick 3 pro Cine for photogrammetry? Thanks.
I used a Mavic 2 for this model. I have used a number of different drones for photogrammetry in the past, but haven't tried the Mavic 3 yet, although I don't think the Cine model adds anything particularly useful for traditional photogrammetry.
Hi! Amazing information! I think you can help me with some information. I have point clouds from a BLK360 scanner from leica. Do you know if it is possible to turn these point clouds into Gaussian Splatting? Thanks a lot!
no, probably not, because the Gaussian splatting is based on image data, not LiDAR data. You could use a photogrammetry program to align your LiDAR datasets to imagery, though.
thank you very much for this video :D superb for future developments in archaeology. I am already checking NERF with Kiri Engine, but I loved to see how to do it with the tools you mentioned. Coool!
.. The building STyle in which were as a daughTer in The KinderGarden is called Villa i had age 5
Hi Matt, are you testing with NV Omniverse? Great video!
No. I tried it about a year ago but didn't find it very compelling.
I wonder how far one could get with enhanced texturing of the ground based on satellite and photosphere images from Google maps (like in MSFS 2020). Cuz otherwise you'd need to get approval to conduct a drone flight over the historical site to get that extra imagery and detail reference.
I’ve thought about screenshot-ing Google earth to create the rooftops/fill architecture, but the quality just isn’t high enough- I think what you’d want to do is get some reference materials for the paving/ground and model that by hand, then PBR texture it
Where is the panorama sphere texture coming from that you used in the end?
It’s a panorama I took using a UAV directly above church rock.
Hi Matthew! Thanks for this video. I'm new and a total noob to this field. I'm a Marketer and my line of work I'm having to learn a lot of these things including reality capture, photogrammetry, NeRFs, 3D GS, Digital Twin. Do you have videos that are educational about these aspects which would help a beginner like me to understand the basics?
The splatter is really cool looking. Looks like a hallucination or a dream
how it accurate compare to epic realitycapture? You talking like old photogrammetry not exist and this is something new and something good, but how it's good? It's requires highend videocard! and result is not better than old photogrammetry what run on old PC just fine.
Not sure I understand what you're asking. NeRF/3DGS are completely different from photogrammetry, the only similarity is that they use the initial camera pose estimation. NeRF/3DGS at the moment (afaik) don't have a quantifiable accuracy and shouldn't be used for anything beyond visualization.
I'd say, if your using reflective or translucent surfaces,i 100% recommend nerfs instead of photogrammatry, cuz it does it somuch better
I made another video explicitly comparing the two- the photogrammetry model actually turned out pretty well.
Well described ❤ Appreciate ❤
Hello Matthew, thank you very much for your response. I assume I need to create an mp4 file if I want to scan a gaussian splatter. What are the settings? I am really a novice in this. Previously I was doing an fps 30. Is it convenient to set fps60? Thanks in advance.
In my experience, still images (photographs) work much better than video! Follow good photogrammetric practice for capture, and then process as a 3DGS.
Great video, thank you. Could you advise me how to scan an object very sharply for gaussian splatting? As it happens, I can't get it sharp. The letters are not clear. I own an android (s22 samsung, nikon z50, insta 360 x3). Which one would be the most accurate? As I am using the texture for training purpose. Thanks in advance. Greetings, Sebas
The Nikon z50 would likely be the best (physical shutter + megapixels), although of course it depends on what lens you are using. In my experience, I get the best results using a high-resolution mirrorless camera (compared to an iphone or action camera). Of course - more data = longer processing times, so there is always a trade off or compromise.
Is it possible combining multiple cameras to get a better quality and keep the 360 capturing workflow speed?
If you had multiple 360 cameras you could mount them vertically on a pole and move that through the scene, yes (getting numerous heights would be key). Another solution would be multiple mirrorless cameras mounted at 3 heights, so something like 12-15 cameras total, firing simultaneously as you moved through an environment.
@MatthewBrennan oh, I forgot to specify, but I was thinking about regular cameras (non-360). My bad. But thanks for answering, I might try it one day for a long shot with no cuts.