Using Kinect to generate SL animation (bvh) files - SLUniverse Forums
Navigation » SLUniverse Forums > Development Discussion and Support > Content Creation » Using Kinect to generate SL animation (bvh) files


Content Creation Discuss building, scripting, and other forms of content creation for SL.

 
Sponsor:
Miriam Chung Estates - Most Sought Land
Reply
 
LinkBack Thread Tools Display Modes
Old 02-24-2011, 04:30 AM   #1 (permalink)
bivirtual and proud of it
 
Rock's Avatar
I started out with nothing, and I still have most of it left :(
 
Join Date: Apr 2009
Location: Location: Location
Posts: 288
SL Join Date: Way back
Using Kinect to generate SL animation (bvh) files

I have been having some success creating bvh files for SL using the Kinect. If anyone else is experimenting in this area and would like to swap experiences or collaborate, just PM me.

Rock
__________________
Chapter & Metaverse Blog: Chapter & Metaverse

Rock is offline   Reply With Quote
Old 02-24-2011, 09:15 AM   #2 (permalink)
nom nom nom
 
Ayu Sura's Avatar
shrink meh
 
Join Date: Aug 2007
Location: under the sea
Posts: 5,828
My Mood:
oooo... nice idea
__________________
"You broke the rules! Now I pull out all your pubic hair!" - Tenacious D

Ayu Sura is offline   Reply With Quote
1 User Agreed:
Old 02-24-2011, 10:12 AM   #3 (permalink)
Cap'n Motorboat

*SLU Supporter*
 
Stroker Serpentine's Avatar
Taint-alizer
 
Join Date: Oct 2007
Location: Cigar City
Posts: 1,360
My Mood:
SL Join Date: Nov 2003
Business: Strokerz Toyz/Strocap/Eros
Send a message via Yahoo to Stroker Serpentine Send a message via Skype™ to Stroker Serpentine
kewl beans..pinged!
Stroker Serpentine is offline   Reply With Quote
Old 02-24-2011, 11:49 AM   #4 (permalink)
Haggis Wrangler

*SLU Supporter*
 
Mojo Skytower's Avatar
Wibble!
 
Join Date: Feb 2011
Location: Aloha!
Posts: 2,190
Business: .::Poise::. Poses
I had heard there was software out ther for doing this but haven't been able to track it down Any pointers on how it works? I thought it sounded as if it could be a lot of fun (or incredibly embarrasing depending on what animations you are trying to make )
__________________
"For me, socks are like sex. Tons of em around and I never seem to have any"
-:-
“It's always funny until someone gets hurt. Then it's just hilarious.”
Mojo Skytower is offline   Reply With Quote
1 User Laughed:
Old 02-24-2011, 11:56 AM   #5 (permalink)
Senior Member
 
Join Date: Sep 2007
Posts: 3,010
Rock, I'd really appreciate hearing more about what it's possible to do. It's visual, so does it just capture movement seen from the front? Does the software use clues in the vision to sort of interpolate limb movement in 3D? How much "fixing up" is required? Does it capture movement across the floor, or just the person's movement relative to themselves?

Can you capture more than one simultaneously, & generate two separate bvh files?
Nika Talaj is offline   Reply With Quote
Old 02-24-2011, 12:23 PM   #6 (permalink)
Noustic Chromomancer
 
Draconis Neurocam's Avatar
 
Join Date: Oct 2010
Posts: 68
My Mood:
SL Join Date: 10/29/2005
Business: Neurocam Industries
Client: viewer-development, NaCl
Blog Entries: 1
Send a message via Skype™ to Draconis Neurocam
Lot of explanation of what Kinect can do: Kinect - Wikipedia, the free encyclopedia

The thought I had though originally, was could the software used to sense everything take a video not created with the hardware and give decent output. The possibilities of such would be very interesting.
__________________
The revolution will be digitized.
Draconis Neurocam is offline   Reply With Quote
Old 02-24-2011, 02:00 PM   #7 (permalink)
bivirtual and proud of it
 
Rock's Avatar
I started out with nothing, and I still have most of it left :(
 
Join Date: Apr 2009
Location: Location: Location
Posts: 288
SL Join Date: Way back
I have posted a full step-by-step tutorial, showing where I am up to right now. Check my signature.

Rock
Rock is offline   Reply With Quote
4 Users Said Thanks :
Old 02-24-2011, 02:27 PM   #8 (permalink)
Hive Mind Director
 
Cristiano's Avatar
Driver Roll Up The Partition Please
 
Join Date: Jun 2007
Location: Miami, FL
Posts: 24,842
My Mood:
SL Join Date: Dec 2002
Business: ANOmations
Client: Viewer 2
Blog Entries: 18
Send a message via Yahoo to Cristiano Send a message via Skype™ to Cristiano
Holy shit, this is huge. Thank you, Rock!
__________________
When the rapture comes, can I have your stuff?



Cristiano is offline   Reply With Quote
Old 02-24-2011, 03:24 PM   #9 (permalink)
Senior Member
 
Winter Ventura's Avatar
 
Join Date: Sep 2007
Posts: 1,835
This sounds interesting, and I'd consider investing in the hardware required, but I'm not sure there's much of a market for avatar animations of "sitting in front of the computer with bad posture"
Winter Ventura is offline   Reply With Quote
1 User Agreed:
Old 02-24-2011, 03:42 PM   #10 (permalink)
Junior Member
 
Twisted Pharaoh's Avatar
Along with Athena, move also your hand
 
Join Date: Feb 2010
Posts: 2,977
My Mood:
SL Join Date: Yesterday
There are a few issues but it's encouraging. (I need a little more testing before commenting)

Last edited by Twisted Pharaoh; 02-24-2011 at 03:51 PM.
Twisted Pharaoh is offline   Reply With Quote
Old 02-24-2011, 03:51 PM   #11 (permalink)
Senior Member
 
Join Date: Sep 2007
Posts: 3,010
Quote:
Originally Posted by Rock View Post
I have posted a full step-by-step tutorial, showing where I am up to right now. Check my signature.
Signatures ... hmmmm.... *turns them on and browses a thread or two *

MAKE IT STOP!!!!


Ah, there it is. Tyvm!
Nika Talaj is offline   Reply With Quote
Old 02-24-2011, 05:06 PM   #12 (permalink)
2つの文化の間立ち往生

*SLU Supporter*
 
Kirari Kujisawa's Avatar
Rock On!
 
Join Date: Sep 2008
Location: Canada
Posts: 12,273
My Mood:
SL Join Date: Jan. 2008
Business: K-10
i read somewhere about microsoft coming out with a PC version of Kinect .

would be cool for live musicuans who could play in front of a kinect and have thier avatars mimic them.
__________________
Kirari's snaps:



Kirari Kujisawa is offline   Reply With Quote
Old 02-24-2011, 05:46 PM   #13 (permalink)
bivirtual and proud of it
 
Rock's Avatar
I started out with nothing, and I still have most of it left :(
 
Join Date: Apr 2009
Location: Location: Location
Posts: 288
SL Join Date: Way back
Quote:
Originally Posted by Kyllie Wylie View Post
i read somewhere about microsoft coming out with a PC version of Kinect .

would be cool for live musicuans who could play in front of a kinect and have thier avatars mimic them.
I believe that has already been done, using the same API that I have been using.

Rock
Rock is offline   Reply With Quote
Old 02-25-2011, 08:19 AM   #14 (permalink)
I am the Script Police!

*SLU Supporter*
 
NeoBokrug Elytis's Avatar
post_count++;
 
Join Date: Sep 2007
Location: Austin, TX
Posts: 1,320
My Mood:
SL Join Date: 11/01/2005
Business: The Wastelands
Client: Official Newest
Is microsoft going to see another spike in Kinect sales this week? Maybe.
NeoBokrug Elytis is offline   Reply With Quote
Old 02-25-2011, 08:55 AM   #15 (permalink)
Junior Member
 
Twisted Pharaoh's Avatar
Along with Athena, move also your hand
 
Join Date: Feb 2010
Posts: 2,977
My Mood:
SL Join Date: Yesterday
I haven't tested it extensively yet, but so far I find it's very good for the natural movement of the head and the hands. Unfortunately I have had issues with arms and legs, I need to check further when I have some time.

Since it captures your avatar when it is facing the camera the side poses wil be likely to give funny results I guess. To correct that you'd need more than one kinect.

However, it's definitely a good ground for a natural pose, even if you need to do some corrections manually.

I can see a number of AO standing poses coming from this but that will be about it (without work I mean).
Twisted Pharaoh is offline   Reply With Quote
Old 02-25-2011, 02:48 PM   #16 (permalink)
bivirtual and proud of it
 
Rock's Avatar
I started out with nothing, and I still have most of it left :(
 
Join Date: Apr 2009
Location: Location: Location
Posts: 288
SL Join Date: Way back
Quote:
Originally Posted by Twisted Pharaoh View Post
I haven't tested it extensively yet, but so far I find it's very good for the natural movement of the head and the hands. Unfortunately I have had issues with arms and legs, I need to check further when I have some time.

Since it captures your avatar when it is facing the camera the side poses wil be likely to give funny results I guess. To correct that you'd need more than one kinect.

However, it's definitely a good ground for a natural pose, even if you need to do some corrections manually.

I can see a number of AO standing poses coming from this but that will be about it (without work I mean).
There is more experimenting going on this weekend. I shall be looking at the full range of controls in the Brekel GUI, and looking more closely at other functions in the OpenNI API. Two other engineers will be looking at stitching two-person anims together (Brekel is working on a new release in which two-persons anims will be handled natively), and all next week will be devoted to exploring the use of 2 or more Kinects.

Fun days ahead

Rock
Rock is offline   Reply With Quote
1 User Said Yay!:
1 User Said Thanks:
Old 02-25-2011, 02:55 PM   #17 (permalink)
I am the Script Police!

*SLU Supporter*
 
NeoBokrug Elytis's Avatar
post_count++;
 
Join Date: Sep 2007
Location: Austin, TX
Posts: 1,320
My Mood:
SL Join Date: 11/01/2005
Business: The Wastelands
Client: Official Newest
Rock: Thank you for paving the way and documenting this sort of development.

When I saw Kinect used for the first time for MoCap, I only knew it was a matter of time before someone would adapt it for SL.
NeoBokrug Elytis is offline   Reply With Quote
1 User Agreed:
Old 03-02-2011, 08:22 PM   #18 (permalink)
Junior Member
 
Twisted Pharaoh's Avatar
Along with Athena, move also your hand
 
Join Date: Feb 2010
Posts: 2,977
My Mood:
SL Join Date: Yesterday
OK, just looked a bit more. I knew something was wrong with arms. BVHacker renames 2 joints lShldr and 2 others rShldr.

What you should do is rename LeftShoulder into lCollar manually, RightShoulder in rCollar then do the Attempt SL Joint naming.

It's not perfect yet but at least the arms move now.
Twisted Pharaoh is offline   Reply With Quote
Old 03-03-2011, 12:48 AM   #19 (permalink)
The Wrong Hands
 
Tizzers Foxchase's Avatar
Naughty FIC
 
Join Date: Jan 2008
Location: Hollywood, Ca
Posts: 112
My Mood:
SL Join Date: Dec 1, 2005
Send a message via AIM to Tizzers Foxchase Send a message via Skype™ to Tizzers Foxchase
I took a trip to Frys and bought a Kinect to try Rock's tutorial. It works surprisingly well!

It takes a few minutes to capture in Brekel and convert using BVHacker's built in SL retargeting tools (without any editing or cleanup). Much easier than the Xsens inertial mocap system that Loki Clifton and I were futzing with last year. Fun stuff.
Tizzers Foxchase is offline   Reply With Quote
1 User Said Yay!:
1 User Said Thanks:
Old 03-03-2011, 03:52 AM   #20 (permalink)
& the voices in his head
 
Evil String's Avatar
Don't Worry Twice
 
Join Date: Mar 2009
Posts: 1,573
I think it is a great excuse to have sex. Convincing .. "hey honey, this is not sex, we are just doing business here "

and if your kinect is turned off ... who cares ?
Evil String is offline   Reply With Quote
Old 03-03-2011, 07:44 AM   #21 (permalink)
SUPER BANNED

*SLU Supporter*
 
Free Xue's Avatar
It's Chinese.
 
Join Date: May 2009
Location: USA! USA! USA!
Posts: 35,545
My Mood:
SL Join Date: May, 2008
Client: V3 w/Starlight
Blog Entries: 10

Awards: 1
Special Achievement in Thread Titling 
Send a message via Skype™ to Free Xue
Quote:
Originally Posted by Evil String View Post
I think it is a great excuse to have sex. Convincing .. "hey honey, this is not sex, we are just doing business here "
Who else uses that for a sexual "transaction". Hmm.
Free Xue is offline   Reply With Quote
2 Users Laughed:
Old 03-25-2011, 02:09 PM   #22 (permalink)
Junior Member
 
deepgreenseas's Avatar
 
Join Date: Mar 2011
Posts: 4
Hi, just recently bought a Kinect and tried following the directions above, but have run into some trouble. I'm using Brekel Kinect v0.38 to create a bvh file using the Second Life bvh capture mode (new to this version of Brekel Kinect?). I then bring the bvh into bvhacker 1.8.0.3. I renamed the joints under the Hack menu, and that checked out. The animation played correctly, but when I imported it into SL, it turned into scrambled eggs. I went back to bvhacker and tried a number of things: I noticed that the abdomen joint was missing, so added that successfully, and ran the retarget to SL under Hack. I also added a T pose frame at the start. Ran again in bvhacker, and all looked just fine. But again, when I brought into SL, my avatar kind of floated oddly in the air, with arms straight out to the sides doing weird leg and abdomen contortions having nothing to do with what I saw in bvhacker. So, I'm at a total loss what I'm doing wrong. Any ideas?
deepgreenseas is offline   Reply With Quote
Old 03-26-2011, 11:29 AM   #23 (permalink)
Junior Member
 
Twisted Pharaoh's Avatar
Along with Athena, move also your hand
 
Join Date: Feb 2010
Posts: 2,977
My Mood:
SL Join Date: Yesterday
Quote:
Originally Posted by deepgreenseas View Post
But again, when I brought into SL, my avatar kind of floated oddly in the air, with arms straight out to the sides doing weird leg and abdomen contortions having nothing to do with what I saw in bvhacker. So, I'm at a total loss what I'm doing wrong. Any ideas?
Quote:
Originally Posted by Twisted Pharaoh View Post
BVHacker renames 2 joints lShldr and 2 others rShldr.

What you should do is rename LeftShoulder into lCollar manually, RightShoulder in rCollar then do the Attempt SL Joint naming.
I haven't tested it with the latest version of bvhacker though. You need to do the Attempt SL joint naming anyway, I guess.
Twisted Pharaoh is offline   Reply With Quote
Old 03-26-2011, 01:43 PM   #24 (permalink)
Just call me Beth
 
Aribeth Zelin's Avatar
Singing along with old music
 
Join Date: Mar 2011
Location: Out in the mists
Posts: 6,571
My Mood:
SL Join Date: Oct 4 2009
Client: Firestorm
Okay, I've got to try this. I've done a few by hand animation/poses but they are so tedious to do and mocap animations seem to sell for better......and I've had a kinect since the holidays and its just gathering dust.... so, if I can get animations from it.......schweet.
__________________
Atomic Faery
Aribeth Zelin is online now   Reply With Quote
Old 03-26-2011, 07:26 PM   #25 (permalink)
Junior Member
 
deepgreenseas's Avatar
 
Join Date: Mar 2011
Posts: 4
Thanks Twisted, but that didn't seem to make any difference. Something is going wrong when I save the file. It looks normal in bvhacker, but if I reload a file I have saved with bvhacker, it no longer works as it did before: the shoulders appear locked, though the legs look fine. But in SL, NOTHING looks normal; it's just a bizarre contortion with no resemblance whatsoever to what it looked like in bvhacker.
deepgreenseas is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On