Interactive Dynamic Video - Realistically Manipulate Objects From Still Frames - General

[hd]https://www.youtube.com/watch?v=4f09VdXex3A[/hd] Clip at 2:00 is mindblowing

can't wait to make the girls tits jiggle in a porno with my mouse

This could be applied to serious engineering applications, like health monitoring of bridges. But what would happen if a cute little alien plopped right down on that there bridge? Stunning.

As a civil engineering student, this is fucking beautiful Probably a long way to go before it's actually accurate enough to study structures, though

examining the vibrations of objects in videos is interesting as fuck. there was an experiment done where they were able to actually pull audio from a video purely by examining the vibrations. [video]https://youtu.be/FKXOucXB4a8[/video]

Oh my god, I want it. I want it, SourceForge that shit, sell it, I'll fucking take it man, I need this in my life! This is SUCH an awesome solution for CG interaction! I'd been trying to use photogrammetry lately to capture foliage, but it's really only good for well-defined solid objects. Plants just have too much detail for a poisson surface reconstruction, it just makes a blobmesh mess, and really accurate CG plants are hardware intensive. Plus, the plants I'm looking for aren't in most CG libraries, and it really often makes things look fake. This is the best of both worlds, dynamics of real, actual scene elements in a simple way. It sounds silly, but without this a CG shot just looks really obvious. Like, there's howling wind or an explosion and nothing REAL in the scene reacts. You either need expensive CG elements that have a high chance of looking awful and fake, you get high-powered fans or leaf blowers, or you have a bunch of people strategically yank on tree-branches and bushes with wires, which isn't always possible or ideal. Oh my god, PLEASE tell me they're going to make this publicly available somehow, this is a GODSEND for low-budget filmmakers.

what if this was applied to a photosphere you can interact with in VR

This is why I love CS. I want to create something as cool as this one day.

Reminds me of this. [Media]https://www.youtube.com/watch?v=e9ASH8IBJ2U[/media]

[QUOTE=Ninja Gnome;50914873]examining the vibrations of objects in videos is interesting as fuck. there was an experiment done where they were able to actually pull audio from a video purely by examining the vibrations. [video]https://youtu.be/FKXOucXB4a8[/video][/QUOTE] Wasn't the feasibility of this proven to be far too low?

[QUOTE=Map in a box;50917503]Wasn't the feasibility of this proven to be far too low?[/QUOTE] What do you mean? The video proves that it's feasible, you just need a high speed camera for best results.

A high speed, ultra high definition, and very high bitrate camera and loud sounds and very low SNR ratio.

[QUOTE=Map in a box;50917605]A high speed, ultra high definition, and very high bitrate camera and loud sounds and very low SNR ratio.[/QUOTE] this is good to have for the best possible quality but according to their research paper you can achieve a level of recovery using a normal digital camera

[QUOTE=Map in a box;50917605]A high speed, ultra high definition, and very high bitrate camera and loud sounds and very low SNR ratio.[/QUOTE] I don't really consider 704x400 to be ultra high definition v:v:v [editline]20th August 2016[/editline] Their high speed camera isn't exactly top of the line.

[QUOTE=Ninja Gnome;50914873]examining the vibrations of objects in videos is interesting as fuck. there was an experiment done where they were able to actually pull audio from a video purely by examining the vibrations. [video]https://youtu.be/FKXOucXB4a8[/video][/QUOTE] vibrations are the future, man

so could you in theory have multiple cameras and use this on an object to create a 3d model with physics built in? if so this could be a real game changer in any modeling industry.

[QUOTE=yodaman888;50918924]so could you in theory have multiple cameras and use this on an object to create a 3d model with physics built in? if so this could be a real game changer in any modeling industry.[/QUOTE] The modelling techniques depend on what is easiest to capture. I think traditional scanning looks at how light reflects off of surfaces and then it maps the reflections, and this method uses high framerate capture to detect subtle changes in the light reflections (vibrations). This technology is a few steps ahead of being revolutionary, only because high framerate cameras are not yet commercially common. First you need a camera that can capture and store the data, and if you think about smartphones, data storage is normally limited to 8-128gb. Its certainly doable, but high speed video eats up data really fast. The next part is that if you can slap a high speed camera onto something common like a phone, you need a lot of light and camera stability to make sure you can see the object you are recording, and that the only vibrations detected in the video are those of the object you are recording. An example with the wind tunnel bridge is that the camera is outside of the wind tunnel looking through a window. If the camera were inside the tunnel and experiences vibrations, you would have to first correct for the camera vibrations before you analyze the object vibrations. The HSM applications are very real because it would be another non-destructive inspection method of structures. If you can analyze a bridge's integrity with an expensive high speed camera and only need to record for a minute in broad daylight, that's very economical. Traditional inspections require part of the bridges to be blocked off causing traffic congestion, and inspecting different parts of a bridge is a real time consuming and medium/high-risk process. The movie applications are neat as well, but perhaps not a game changer (yet). The low budget interactions seem too cheap for big studios to use as a CGI replacement but still too expensive in equipment and environment requirements for small studios and indie filmmakers to utilize.

[media]https://www.youtube.com/watch?v=98E0FNOch0w[/media] would be interesting to try and print the jiggle of the world, if that makes sense.