FlexPoser: Real-time face tracking and animation [Public testing phase]

[NisshokuZK]

Thank you kindly. :)

I am aware of a lot of issues about webcam compatibility and some crashes (crashing when pressing Enable and crashing when trying to create a profile, both which are presumable related with webcam compatbility). I'm looking at these issues and will release an update next Thursday. The server addon will be updated automatically via Workshop, but you will have to download the new version of the DLL then (you will be notified in-game if your DLL is outdated).

[Mista Tea]

Is there any proper place that I should report errors to?

I've installed this on my Sandbox server, multiplayer, and it appears that as certain clients load into the server, they are spammed with the following error:

[ERROR] lua/shared_apply_weights.lua:3: attempt to index local 'ent' (a boolean value)
  1. ApplyWeights - lua/shared_apply_weights.lua:3
   2. func - lua/shared_net.lua:85
    3. unknown - lua/includes/modules/net.lua:32

It looks like the error may also occur for random clients even after they've initalized in the server.

Also, terrific work. I'm going for my Master's degree in Computer Science this fall and seeing that you've done this for your own studies has really encouraged me on the possibilities. Cheers :)

[Scott]

[img]http://i.imgur.com/e3WhCZgl.jpg[/img]

Great work!

[Lyokanthrope]

Hm. I can't get this working at all. Crashes no matter what webcam I use...

[NisshokuZK]

I'm working on a new build that I would like testers for, preferably people who are experiencing crashes right now. If you would like to help, please PM me or reply here and I'll send you a Dropbox link containing a new version of the server addon, the DLL and some test instructions.

[circuitbawx]

I somehow tilted my head and turned my neck to one side and my character in game decided to bend his neck back to almost 90 degrees and this expression popped on his face out of nowhere

[IMG]http://i.imgur.com/wBfoA7J.jpg[/IMG]

I was scared

[harrisonj0122]

Ya, i need the new version, the dropbox link. My game crashes whenever i enable it.

[HonkAbby]

Very awesome work! I am impressed, this is something I find very needed when it comes to virtual reality and roleplaying!
I've showed this to a few friends and they're equally impressed! Very eager to try it out myself. I'll see if it's compatible with my integrated webcam.

Keep up the great work, good luck on your Masters!

[Skulman54]

Talking here on my Phone at 7:00 AM, I decided that it would be a great idea to show you a neat website to track what servers are using your addon. [url]http://www.gametracker.com/search/?search_by=server_variable&search_by2=Fp_trackingenabled&query=&loc=_all&sort=&order=[/url] Also, Typing and formatting on an IPhone is REALLY hard.

[NisshokuZK]

Skulman54, thats a neat link! Cool to see so who is using the addon. :D

Everyone, I just updated the server addon and the binary module. Server addon should be updated via Workshop, and users will need to redownload the DLL from flexposer.com/download.php. I've put the change notes in the opening post. If your server runs an up-to-date server addon, then players will be notified if their DLL is outdated. However, if your server addon is outdated and a player joins with the new DLL, problems will occur so update the addon ASAP. I've prepared checks in the latest version of the server addon so that this won't be a problem for future updates.

[Dancingonpie]

My webcam worked fine with this addon before the update, now when I select it through the devices list, the webcam's "on" light blinks three times and the webcam does nothing. What might be the cause?

[NisshokuZK]

Hey, did you press the Refresh device list button just before the blinking? Currently to test for webcams I attempt to connect to a device, read an image from it and then close it. Its a cheat in a way, but also necessary to distinguish webcams from card readers since OpenCV seems to detect card readers as video capture devices. Anyway the short poking results in blinking and some webcams are actually registered to multiple slots which results in multiple blinking.

However, if you see the blinking when actually selecting the camera... that is a bug. And if you can't get your camera to work right now, sorry apparently I messed something up. I'll need to take a look at it tomorrow. I posted another fast update a few minutes ago, which probably doesn't solve the problem but does include client-side logging that is useful to me. You can help me out by going to the GarrysMod/garrysmod/lua/bin folder right after a crash and emailing the fp_log.txt file to zhikang.research [at] gmail [dot] com (does this actually work against spambots? no matter).

[TomatoSoup]

Earlier today this worked for me. It then told me I had to update and, after doing so, it now crashes every time I try to Enable. The webcam's light flickers on briefly before Garrysmod freezes, then closes.

Also, I'm sorry to tell you but I do not have a lua/bin/fp_log.txt file.

[HonkAbby]

[QUOTE=TomatoSoup;45382005]The webcam's light flickers on briefly...[/QUOTE]

Same here, although mine doesn't crash - it just says 'the device can't be accessed'.
In some cases I can get the FlexPoser to register it, but no white rectangle appears to set-up.
Running as admin also seems to have no effect.

[Erkor]

I just tested the new build. The camera thing [I]should[/I] be fixed. It works for the Microsoft LifeCam HD-3000 if you remove the software. It seems to disconnect the camera.
[B]However[/B], the camera doesn't turn on. It only gets selected.

[NisshokuZK]

Edit: Just uploaded a new version of server addon and DLL. This should fix issues that were introduced by yesterday's update. Special thanks to [B]erkor[/B] for trying out ~10 test builds, which helped me localize the issue.

[MrNegaBlox]

I am getting "could not access this device" can someone tell me how to fix this?

Thank you in advance!

[TomatoSoup]

A player just had a driver crash and was dropped from my TTT server after being killed. FlexPoser had an error:

[FlexPoser: Real-time face tracking and animation] lua/shared_apply_weights.lua:3: attempt to index local 'ent' (a boolean value)
  1. ApplyWeights - lua/shared_apply_weights.lua:3
   2. func - lua/shared_net.lua:85
    3. unknown - lua/includes/modules/net.lua:32

Although, because it was only me and my friend, I don't know if FlexPoser actually broke or not.

[HonkAbby]

Great! The updated fixes it.

Will there be any way to be able to use the voice function (which the player character opening their mouth according to your mic speech levels) and the expressions of Flexposer at the same time?
I played around with it and the face went into the neutral expression while talking.

[NisshokuZK]

HonkAbby, glad to hear the update solved your issues. One less worry for me. :P

MrNegaBlox (and others with webcam issues), FlexPoser uses the library OpenCV for accessing webcams. I recently made a small tool that allows you to select a webcam and then tries to access it using OpenCV. It will then tell you whether your webcam + drivers in their current state are compatible with OpenCV (and thus, FlexPoser) or not. That should make problem solving a little easier, since you don't have to boot Garry's Mod to test. You can download the tool here: [url]www.flexposer.com/fpWebcamTest.zip[/url]. Just like for FlexPoser, you need VC++ 32-bit redists to use it.

These ideas might help if they are applicable to you:
- Download latest webcam drivers
- Make sure your webcam is enabled in device manager
- Disable webcam splitting software in device manager, or other drivers/services that may "claim" webcams

There is also the possibility that your webcam is not supported by OpenCV, then you are out of luck.

TomatoSoup, thanks for reporting, I forgot about this issue. Luckily it is non-critical, i.e. it doesn't affect further execution of FlexPoser. Its caused by a player sending his facial data to the server and then disconnecting before other clients have the opportunity to apply the facial data to that player's model (but he left, so player.GetByID returns false). I'll fix this and update the server addon once I have the opportunity. No DLL redownload needed. :)

[Phoenixf129]

[QUOTE=NisshokuZK;45398515]HonkAbby, glad to hear the update solved your issues. One less worry for me. :P

MrNegaBlox (and others with webcam issues), FlexPoser uses the library OpenCV for accessing webcams. I recently made a small tool that allows you to select a webcam and then tries to access it using OpenCV. It will then tell you whether your webcam + drivers in their current state are compatible with OpenCV (and thus, FlexPoser) or not. That should make problem solving a little easier, since you don't have to boot Garry's Mod to test. You can download the tool here: [url]www.flexposer.com/fpWebcamTest.zip[/url]. Just like for FlexPoser, you need VC++ 32-bit redists to use it.

These ideas might help if they are applicable to you:
- Download latest webcam drivers
- Make sure your webcam is enabled in device manager
- Disable webcam splitting software in device manager, or other drivers/services that may "claim" webcams

There is also the possibility that your webcam is not supported by OpenCV, then you are out of luck.

TomatoSoup, thanks for reporting, I forgot about this issue. Luckily it is non-critical, i.e. it doesn't affect further execution of FlexPoser. Its caused by a player sending his facial data to the server and then disconnecting before other clients have the opportunity to apply the facial data to that player's model (but he left, so player.GetByID returns false). I'll fix this and update the server addon once I have the opportunity. No DLL redownload needed. :)[/QUOTE]

[vid]http://cybergmod.net/i/TeNtCV.mp4[/vid]
What if it crashes? I am doing things a little unauthodox here though, using an application called iWebCamera (iPhone software) so I use my iPhone as a webcam.

[NisshokuZK]

[QUOTE=Phoenixf129;45409117][vid]http://cybergmod.net/i/TeNtCV.mp4[/vid]
What if it crashes? I am doing things a little unauthodox here though, using an application called iWebCamera (iPhone software) so I use my iPhone as a webcam.[/QUOTE]

If it crashes its not compatible. It won't crash in-game because in the DLL the exception is caught, but the camera can't be used by FlexPoser.

[rainbow-dash]

This is actually a pretty fucking cool. I'm a nub when it comes to binary modules but I imagine something like this is probably very complex.

[thoy450]

Having some trouble with this.

Put the files in the right folder:
[url]http://imgur.com/3ukn7Xk[/url]

Have the correct version of Microsoft Visual C++ 2012 installed:
[url]http://imgur.com/tOSahW8[/url]

This is the error I get in console:
[url]http://imgur.com/38Ih6QA[/url]

[GameWalrus]

I envy the fact that anyone could make something like this for GMod.

[editline]5th August 2014[/editline]

I'm sorry I had to make a video of this;

[video=youtube;EGLhISBEd4E]http://www.youtube.com/watch?v=EGLhISBEd4E[/video]

[Catnipbuddy]

[QUOTE=NisshokuZK;45398515]I recently made a small tool that allows you to select a webcam and then tries to access it using OpenCV.[/QUOTE]
I used that tool and it says that its compatible after I downloaded the new DLL's that came out recently and put them in the correct folder, but when I select my webcam ingame, it freezes, then I tab out and see a window that pops up and asks me to choose a device, it is already on my webcam, so I hit Ok, go back in game, and it says "Cannot access this device"

[Speedspaz]

For anyone who are having issues with the missing .dll files, did what I did.

1) Delete all other C++ Redistributable Packaces and only install the 2012 x86 one and then reboot.

2) If still dont work copy the binary(.dll) files to every possible bin folder.

garrysmod/garrysmod/lua/bin
garrysmod/garrysmod/bin
garrysmod/bin

It worked for me! ;)

[Muura]

Fucking beautiful, love to know that a game like this can get an addon like this while other people are trying to do the same thing on the next-gen and failing at it.

[ricky23]

Not bad, used my phones camera

[Eloquence]

Epic. Just epic.

Any idea of the effect it would have on traffic and resources per person?