AI Learns Human Movement From Unorganized Data 🏃‍♀️


Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. Last year, an amazing neural network-based
technique appeared that was able to look at a bunch of unlabeled motion data, and learned
to weave them together to control the motion of quadrupeds, like this wolf here. It was able to successfully address the shortcomings
of previous works, for instance, the weird sliding motions have been eliminated, and
it was also capable of following some predefined trajectories. This new paper continues research in this
direction by proposing a technique that is also capable of interacting with its environment
or other characters, for instance, they can punch each other, and after the punch, they
can recover from undesirable positions, and more. The problem formulation is as follows – it
is given the current state of the character and a goal, and you see here with blue how
it predicts the motion to continue. It understands that we have to walk towards
the goal, that we are likely to fall when hit by a ball, and it knows, that then, we
have to get up and continue our journey, and eventually, reach our goal. Some amazing life advice from the AI right
there. The goal here is also to learn something meaningful
from lots of barely labeled human motion data. Barely labeled means that a bunch of videos
are given almost as-is, without additional information on what movements are being performed
in these videos. If we had labels for all this data that you
see here, it would say that this sequence shows a jump, and these ones are for running. However, the labeling process takes a ton
of time and effort, so if we can get away without it, that’s glorious, but, in return,
with this, we create an additional burden that the learning algorithm has to shoulder. Unfortunately, the problem gets even worse
– as you see here, the number of frames contained in the original dataset is very scarce. To alleviate this, the authors decided to
augment this dataset, which means trying to combine parts of this data to squeeze out
as much information as possible. You see some examples here how this motion
data can be combined from many small segments, and in the paper, they show that the augmentation
helps us create even up to 10 to 30 times more training data for the neural networks. As a result of this augmented dataset, it
can learn to perform zombie, gorilla movements, chicken hopping, even dribbling with a basketball,
you name it. What’s even more, we can give the AI high-level
commands interactively, and it will try to weave the motions together appropriately. They can also punch each other. Ow. And all this was learned from a bunch of unorganized
data. What a time to be alive! Thanks for watching and for your generous
support, and I’ll see you next time!

100 Comments

  1. Wavvy said:

    Great work we are literally teaching the terminators how to free-run.

    October 19, 2019
    Reply
  2. Cat said:

    Does it do it in real time?

    October 19, 2019
    Reply
  3. Captain Of A Starship said:

    This could be sweet in gaming and for robotics.

    October 20, 2019
    Reply
  4. Mike Murkovic said:

    Good life lesson from AI: if you get hit by a ball really hard, remember to get up and keep going towards your goal

    October 20, 2019
    Reply
  5. Tim Hampton said:

    “Some amazing life advice from AI right there.”

    October 20, 2019
    Reply
  6. raph151515 said:

    this will end up generating the learning of a real fighter robot

    October 20, 2019
    Reply
  7. maxjt11 said:

    This coupled with Boston Dynamics Atlas robot would be incredible, it could copy video input perfectly

    October 20, 2019
    Reply
  8. Kido Dreauw said:

    Wasn't there a paper about tracking and transferring motion (dancing?). Use that to get tonnes of different movement data, get data from ppl moving just on YouTube or whatever, train all random movements for different situations.

    Then train it on 3d environments approximating real world. Figure out how to transfer this knowledge to real robots(there were papers about this too!) and you're pretty much at Boston dynamics level minus the amazing hardware.

    Maybe the need for amazing hardware can be compensated by optimising for 'careful and conserving' movement when training AI?

    Technology progress is accelerating!! I feel so behind 🙁

    October 20, 2019
    Reply
  9. PseudoSarcasm said:

    1:10 Donny Darko anyone?

    October 20, 2019
    Reply
  10. Sammy Spaniel said:

    Long story short…….20 years from now we will have flying terminators. I'm not trolling. This is serious. Look at the evolution of the automobile from 1905 to 1925. The difference is astounding. We went from dirt roads full of horses to paved roads full of automobile. We are in a similar evolution with robots. Human soldiers on the battle field will be as rare as weaponized blimps soon.

    October 20, 2019
    Reply
  11. manas sharma said:

    What if we could use deep learning for anime making

    October 20, 2019
    Reply
  12. Eric lamb said:

    This video is why I'm subscribed

    October 20, 2019
    Reply
  13. DXM said:

    Wow this is amazing!!! … Or as many of us would say: " *click* nice "

    October 20, 2019
    Reply
  14. Giorgi Sabashvili said:

    why do we fall?

    October 20, 2019
    Reply
  15. Rajib Bando said:

    Wow! Really looking forward to the time when augmented artificial musculo-skeletal system shall help us move real fast without moving a single biological muscle strand. Or embedded BCI help us simulate environments and interact with them.

    October 20, 2019
    Reply
  16. TheAlphazeta09 said:

    The best moment in these videos, is always when you say "What a time to be alive!". Thank you for your optimism! 🙂

    October 20, 2019
    Reply
  17. Hegedus Krisztian said:

    wait are you hungarian?

    October 20, 2019
    Reply
  18. a a said:

    Some applications to robotic simulations are possible with this.

    October 20, 2019
    Reply
  19. Elena said:

    0:44 and they walk away as if nothing happened like in GTA 😂😂😂😂

    October 20, 2019
    Reply
  20. Jayanth Veeru said:

    Where do I learn all this? Do you have any videos on it?

    October 20, 2019
    Reply
  21. Taran Van Hemert said:

    That dog animation was incredible

    October 20, 2019
    Reply
  22. Suhail Alhegry said:

    Where does one find those datasets?

    October 20, 2019
    Reply
  23. Ryae on said:

    i curious and want to feel how playing a RPG game with A.I NPC..
    2020 NPC still passive? static? never move from original place? just walk without goal then vanish when not rendered?
    give them A.I. let the world progress!! if it offline game then no one have same experience because A.I will constantly changing the world.
    merchant NPC? their item should acquired from interaction not spawned (maybe exception if not rendered/out of sight -with history on their data where they get the item-).
    well is this possible with our low/medium end CPU? because not everyone can buy monster PC. (maybe this is the reason we must wait for 2030 for a game implement this)

    October 20, 2019
    Reply
  24. quoc tu said:

    I have been a fan of your channel for a while. Today I clicked on your ads about Lambda GPU cloud and decided to try it. However after I filled in my credit card info and submit, the site just shown an alert "Hi". Very shady if you ask me. After that I tried to launch an instance and it just show error message. I tried to delete my card on the site but cannot find that feature anywhere, also updating the card info did not work either. Just a warning for other if they want to try it 🙁

    October 20, 2019
    Reply
  25. Tapa said:

    Man I am working on Human-Object Interaction Detection. this could be helpful

    October 20, 2019
    Reply
  26. bgtubber said:

    Now implement this in games to use in place of the predefined character animations! They will look so much more realistic.

    October 20, 2019
    Reply
  27. Ojasvi Singh 786 said:

    ✨✨✨🤩🤩🤩

    October 20, 2019
    Reply
  28. DeepSpace12 said:

    2:56 Speaking of Lambda, someone should mod Half-Life 1's hazard course tutorial…

    October 20, 2019
    Reply
  29. Adrien Tasic said:

    Can't wait to watch UFC with 3D AI fighters

    October 20, 2019
    Reply
  30. DeepSpace12 said:

    "Does this unit have a soul?"
    Punch

    October 20, 2019
    Reply
  31. Luis M said:

    Video games are going to be amazing in 5 years! can't wait to see this applied!

    October 20, 2019
    Reply
  32. jinesh jain said:

    This is machine learning not ai😕

    October 20, 2019
    Reply
  33. IKAROS 420 said:

    1:00 reminds me of the feeling of falling on psychedelic drugs, like body cascading through time in frames and loops.
    Crazy stuff

    I can't wait for some AI type games which (who?) can predict and adapt the game to your movements

    October 20, 2019
    Reply
  34. Chance Briggs said:

    Imagine the future. I’ve been ready for a while for what’s to come. I’m guessing five years before it’s officially time to change life as we know it. With All this data being gathering from different outlets, will re write our understanding of what we think is reality.

    October 20, 2019
    Reply
  35. Wizzard said:

    What if we teach AI to be aware of itself.

    Or what if we just teach it to fuck.

    October 20, 2019
    Reply
  36. Oleg Skutte said:

    This would be a great fit for my physics based fighting game 😯
    Can't wait until they release the code

    October 20, 2019
    Reply
  37. Victor Hugo said:

    nice, more AI and time for job to taken is coming sooner, and maybe the AI can harvest me to add hardware and make more powerful

    October 20, 2019
    Reply
  38. Victor Hugo said:

    why the fuck is this dangerous crap AI and ML activity not banned, like make it a criminal activity, and destroy illegal tech/product/virtualware/code or whatever it is, any moron can see the fucking danger and destruction of humanity resulting from it

    October 20, 2019
    Reply
  39. marv1n said:

    sumotori dreams did this 13 years ago smh

    October 20, 2019
    Reply
  40. Growth Mindset said:

    I really enjoy the quality and variety of educational videos on this channel.
    All I can say is keep it coming ! 🙂

    October 20, 2019
    Reply
  41. rice farmer said:

    0:57 this is requiem

    October 20, 2019
    Reply
  42. SuperViperGamers said:

    the future of npcs in videogames are looking good

    October 20, 2019
    Reply
  43. T Minus said:

    The whole time I was just thinking GTA VI

    October 20, 2019
    Reply
  44. david defendini said:

    can someone feed some parkour data to this ?

    October 20, 2019
    Reply
  45. Simon Carlile said:

    1:01 that reminds me of that donnie darko scene where he sees all the blue trails leading people to do what they're about to do

    October 20, 2019
    Reply
  46. Caleb Laws said:

    How far are we from implementing this in real time? I'd be amazing to see a fighting game utilize these animation techniques.

    October 20, 2019
    Reply
  47. Moby Motion said:

    Love anything that helps us get more from our data – wow

    October 21, 2019
    Reply
  48. mtlicq said:

    Teach AI Monty Python's "funny walks" fok AI

    October 21, 2019
    Reply
  49. Taylor Murphy said:

    Though there was no video corresponding to it, there's a recent paper about learning to grasp clear objects. Like empty glasses. Pretty cool stuff

    October 21, 2019
    Reply
  50. THEBIGMEOW said:

    What scares me the most when someone asks AI to fix world problems. Honestly to tell you the truth we need problems in order to progress forward.

    October 21, 2019
    Reply
  51. Anthony Barker said:

    Most useful channel on the internet

    October 21, 2019
    Reply
  52. Telekinesys said:

    0:58 Looks like Atium to me.

    October 21, 2019
    Reply
  53. Felipe Mantovanelli said:

    1:04 LMAO

    October 21, 2019
    Reply
  54. Antonio Stanziola said:

    What's going on with this channel? This video just shows the problem and results, what about the methods? That's what used to make 2MP interesting, now is just turning into a showcase of results. Boring.

    October 21, 2019
    Reply
  55. Derpy said:

    How was that wolf made and put into a game?

    October 22, 2019
    Reply
  56. pronobozo said:

    Update the boston dynamics robot

    October 22, 2019
    Reply
  57. Verify Truth said:

    I think I can train an AI to Love and Seek Truth.

    October 22, 2019
    Reply
  58. D Man said:

    Training environment for robots,

    October 22, 2019
    Reply
  59. 『conroy』 / SlimDaru / HarTH said:

    what if we could code a program that it makes itself think its alive and have consciousness and make its own world have its own rule in physic etc.

    October 23, 2019
    Reply
  60. Max Bulatoff said:

    Soon we will see they to kick each other…
    Then shooting each other with guns…

    October 23, 2019
    Reply
  61. TaliiLars said:

    0:25 Clearly, the weird slighting motion hasn't been eliminated.

    October 23, 2019
    Reply
  62. Kevin Arie said:

    This being used in games would be actually amazing, the movements would be so smooth and real

    October 23, 2019
    Reply
  63. Ultra light said:

    if we were to build a robot that would be the exact wheight and dimensions this AI had for data then he could run around like a human XD

    October 24, 2019
    Reply
  64. Dominus Empyreus said:

    How do they compensate for different frictions and gravities?

    October 24, 2019
    Reply
  65. Jan B said:

    please continue teaching AI how punching works!
    might come in handy for our future cyber overlords lol

    October 24, 2019
    Reply
  66. Tomi Johannes said:

    Finally punching! It can punch!

    October 24, 2019
    Reply
  67. ShkololoCraft said:

    1:00 Epitaphu!

    October 24, 2019
    Reply
  68. Jose Rafael said:

    imagine any open world game with physics like this.

    October 24, 2019
    Reply
  69. gustavobarr said:

    I hope Trump hating NPC's receive this upgrade it will make them look more human.

    October 24, 2019
    Reply
  70. Pea Stone said:

    How about one with a kick in the nuts.

    October 25, 2019
    Reply
  71. CLICKEROFTRUTH said:

    Something something donny darko

    October 25, 2019
    Reply
  72. Baleur said:

    All this AI stuff that's being trained for thousands of hours right now, will be SO amazing for gaming in the coming years.

    October 25, 2019
    Reply
  73. Lavos 1408 said:

    i want more i want they Ai not only recive the punch also feel pain hahaha

    October 25, 2019
    Reply
  74. Rigatonia said:

    Soon enough the Ai will take the coder's job lol.

    October 25, 2019
    Reply
  75. MrJured100 said:

    Holy fuck this is amazing!

    October 25, 2019
    Reply
  76. Bryan Jamison said:

    That dog animation is dope

    October 26, 2019
    Reply
  77. Jay Bro said:

    Paaabalam

    October 26, 2019
    Reply
  78. Taterz said:

    this is terrifying.

    October 26, 2019
    Reply
  79. Alexander Horstkötter said:

    Now put it into a robot!

    October 27, 2019
    Reply
  80. Theodore Reaper said:

    2300 AD – Watches as a robot punches a human and walks away. Best decision ever made.

    October 28, 2019
    Reply
  81. wilwad said:

    Why did he punch, why not hug or shake hands? tsk tsk

    October 29, 2019
    Reply
  82. Brant Wedel said:

    Hold on to your papers, and motion capture your horses!

    October 29, 2019
    Reply
  83. -CFW. Magic- said:

    Hunh… When I tried this the Ai kept rolling to avoid enemy attacks.

    October 29, 2019
    Reply
  84. Glyne Martin said:

    if you have pictures, why do you need labels?. Can there be labels without pictures?

    October 30, 2019
    Reply
  85. Zetsuke4 said:

    Dammn

    October 30, 2019
    Reply
  86. Ipotrick said:

    impressive but the strange slide still persists just to a lower extend

    October 30, 2019
    Reply
  87. Dr. Scientist said:

    This is like an automatic inbetweener… Animators, this will make your life much easier (or possibly make you completely out of a job)

    October 30, 2019
    Reply
  88. freak gamer said:

    Whats that game-like thing at the end with buttons? Is it public? I wanna try it out!

    October 31, 2019
    Reply
  89. Blotted said:

    Remember when Euphoria was doing great animation with their Natural Motion that Rockstar still uses.

    October 31, 2019
    Reply
  90. Leaetta Hyer said:

    WAIT UNTIL AI SAYS IT POURED OVER MILLIONS OF PAGES OF DATA FROM OVER 370 PHARMACEUTICAL COMPANIES AND IT FOUND A FORMULA TO ELIMINATE THE ENTIRE AUTISM SPECTRUM AND BREAST CANCER ! MOST WOMEN TAKE THE DRUG BUT ARE TRICKED. THE DRUG SECRETLY DESTROYS THRIR LIVER AND OVARIES AND ALL DIE IN 6-9 MONTHS

    November 1, 2019
    Reply
  91. Karthik Natarajan said:

    Jesus chirst. They've started punching.

    November 1, 2019
    Reply
  92. tn said:

    gbp clout services?

    November 2, 2019
    Reply
  93. Stellar said:

    4:23 vibe check

    November 3, 2019
    Reply
  94. mark jones said:

    Alpha intelligence…?

    November 6, 2019
    Reply
  95. Auxence Fromont said:

    now could you make them fight each other?

    November 7, 2019
    Reply
  96. Andreo Suro said:

    классно сделаны движения, но почему в играх АИ всегда такой тупой и глючный, не интересный и не разнообразный в поведении?))

    November 10, 2019
    Reply
  97. Generic Internetter said:

    The Euphoria animation engine did this over a decade ago. Why are you talking about this like it’s something new?

    November 12, 2019
    Reply
  98. KAIDZ said:

    one day we will see Elon Musk as the Terminator warning us from the Future.

    November 15, 2019
    Reply
  99. Can Öztürk said:

    beyler bu nedir amk anlamıyorum ama izliyorum

    November 17, 2019
    Reply
  100. sourav agarwal said:

    Can you please share the link to this paper?

    November 26, 2019
    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *