{"id":1000,"date":"2020-07-09T17:07:00","date_gmt":"2020-07-09T08:07:00","guid":{"rendered":"https:\/\/arithmer.blog\/?p=1000"},"modified":"2022-03-08T15:43:45","modified_gmt":"2022-03-08T06:43:45","slug":"3d-human-body-modeling-from-rgb-images","status":"publish","type":"post","link":"https:\/\/arithmer.blog\/blog\/3d-human-body-modeling-from-rgb-images","title":{"rendered":"\u753b\u50cf\u304b\u3089\u306e3D\u4eba\u4f53\u30e2\u30c7\u30ea\u30f3\u30b0"},"content":{"rendered":"\n<p class=\"has-small-font-size\">\u672c\u8cc7\u6599\u306f2020\u5e747\u67089\u65e5\u306b\u793e\u5185\u5171\u6709\u8cc7\u6599\u3068\u3057\u3066\u5c55\u958b\u3057\u3066\u3044\u305f\u3082\u306e\u3092WEB\u30da\u30fc\u30b8\u5411\u3051\u306b\u30ea\u30cb\u30e5\u30fc\u30a2\u30eb\u3057\u305f\u5185\u5bb9\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"outline\"><strong>\u25a0Outline<\/strong><\/h3>\n\n\n\n<ul style=\"font-size:16px\"><li><strong>Introduction<\/strong><ul><li>Definition of the problem<\/li><li>Challenges<\/li><li>Common approaches<\/li><\/ul><\/li><li><strong>Statistical body models<\/strong><ul><li>Definition of the setup<\/li><li>Aim of the methods<\/li><li>Typical solutions<\/li><li>SMPL<\/li><\/ul><\/li><li>3D human pose<ul><li>Challenges<\/li><li>2D solutions<\/li><li>SMPLify<\/li><\/ul><\/li><li>Conclusions<\/li><\/ul>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"the-data\"><strong>\u25a0The data<\/strong><\/h3>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"1024\" height=\"371\" src=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_01.jpg\" alt=\"\" class=\"wp-image-990\" srcset=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_01.jpg 1024w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_01-300x109.jpg 300w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_01-768x278.jpg 768w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_01-304x110.jpg 304w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/figure>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"the-model\"><strong>\u25a0The Model<\/strong><\/h3>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"1024\" height=\"371\" src=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_02.jpg\" alt=\"\" class=\"wp-image-991\" srcset=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_02.jpg 1024w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_02-300x109.jpg 300w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_02-768x278.jpg 768w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_02-304x110.jpg 304w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/figure>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"smpl\"><strong>\u25a0SMPL<\/strong><\/h3>\n\n\n\n<p style=\"font-size:16px\">SMPL stands for Skinned Multi-Person Linear model. <br>It is a \u201cstatistical\u201d body model, which means it is learned from data. <br>It captures 2 aspects of the human body: <\/p>\n\n\n\n<ol><li>Shape<\/li><li>Pose<\/li><\/ol>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"1024\" height=\"165\" src=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_03.jpg\" alt=\"\" class=\"wp-image-992\" srcset=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_03.jpg 1024w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_03-300x48.jpg 300w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_03-768x124.jpg 768w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_03-304x49.jpg 304w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/figure>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"example-joint-regression\"><strong>\u25a0Example: Joint regression<\/strong><\/h3>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"1024\" height=\"362\" src=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_04.jpg\" alt=\"\" class=\"wp-image-993\" srcset=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_04.jpg 1024w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_04-300x106.jpg 300w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_04-768x272.jpg 768w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_04-304x107.jpg 304w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/figure>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"example-inference\"><strong>\u25a0Example: Inference<\/strong><\/h3>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"1024\" height=\"373\" src=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_05.jpg\" alt=\"\" class=\"wp-image-994\" srcset=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_05.jpg 1024w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_05-300x109.jpg 300w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_05-768x280.jpg 768w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_05-304x111.jpg 304w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/figure>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"smplify\"><strong>\u25a0SMPLify<\/strong><\/h3>\n\n\n\n<p>The goal of <strong>SMPLify<\/strong> is to automatically estimate <br><strong>both<\/strong> 3D <strong>pose<\/strong> and 3D body <strong>shape<\/strong> from a <strong>single<\/strong> RGB <strong>image<\/strong>.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"1024\" height=\"296\" src=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_06.jpg\" alt=\"\" class=\"wp-image-995\" srcset=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_06.jpg 1024w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_06-300x87.jpg 300w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_06-768x222.jpg 768w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_06-304x88.jpg 304w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/figure>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"ingredients\"><strong>\u25a0Ingredients<\/strong><\/h3>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"1024\" height=\"359\" src=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_07.jpg\" alt=\"\" class=\"wp-image-996\" srcset=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_07.jpg 1024w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_07-300x105.jpg 300w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_07-768x269.jpg 768w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_07-304x107.jpg 304w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/figure>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"objective-functions\"><strong>\u25a0Objective functions<\/strong><\/h3>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"1024\" height=\"382\" src=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_08.jpg\" alt=\"\" class=\"wp-image-997\" srcset=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_08.jpg 1024w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_08-300x112.jpg 300w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_08-768x287.jpg 768w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_08-304x113.jpg 304w\" sizes=\"(max-width: 1024px) 100vw, 1024px\"><\/figure>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"example-comparison\"><strong>\u25a0Example: comparison<\/strong><\/h3>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"1022\" height=\"348\" src=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_09.jpg\" alt=\"\" class=\"wp-image-998\" srcset=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_09.jpg 1022w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_09-300x102.jpg 300w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_09-768x262.jpg 768w, https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/NS20200709_09-304x104.jpg 304w\" sizes=\"(max-width: 1022px) 100vw, 1022px\"><\/figure>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"references\"><strong>\u25a0References<\/strong><\/h3>\n\n\n\n<p style=\"font-size:18px\"><strong>Models<\/strong><\/p>\n\n\n\n<ul style=\"font-size:16px\"><li><strong>SMPL<\/strong><ul><li>Project website: <a rel=\"noreferrer noopener\" href=\"https:\/\/smpl.is.tue.mpg.de\/\" target=\"_blank\">https:\/\/smpl.is.tue.mpg.de\/<\/a><\/li><\/ul><\/li><li><strong>SMPLify<\/strong><ul><li>Project website: <a rel=\"noreferrer noopener\" href=\"http:\/\/smplify.is.tue.mpg.de\/\" target=\"_blank\">http:\/\/smplify.is.tue.mpg.de\/<\/a><\/li><\/ul><\/li><\/ul>\n\n\n\n<p><\/p>\n\n\n\n<p style=\"font-size:18px\"><strong>Datasets<\/strong><\/p>\n\n\n\n<ul style=\"font-size:16px\"><li><strong>FAUST<\/strong><ul><li>Project website: <a rel=\"noreferrer noopener\" href=\"http:\/\/faust.is.tue.mpg.de\/\" target=\"_blank\">http:\/\/faust.is.tue.mpg.de\/<\/a><\/li><\/ul><\/li><li><strong>CAESAR<\/strong><ul><li>Project website: <a rel=\"noreferrer noopener\" href=\"http:\/\/store.sae.org\/caesar\/\" target=\"_blank\">http:\/\/store.sae.org\/caesar\/<\/a><\/li><\/ul><\/li><\/ul>\n\n\n\n<p style=\"font-size:16px\"><strong>My ongoing notes on Notion:<\/strong><br><a rel=\"noreferrer noopener\" href=\"https:\/\/www.notion.so\/erinaldi\/Fundamentals-of-riggin\" target=\"_blank\">https:\/\/www.notion.so\/erinaldi\/Fundamentals-of-riggin g-eb8acd313a444b6999ef1b0a7a13b892<\/a><\/p>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"future-discussion\"><strong>\u25a0Future discussion<\/strong><\/h3>\n\n\n\n<p style=\"font-size:18px\"><strong>A few links for modern motion capture papers:<\/strong><\/p>\n\n\n\n<ul><li><strong>DeepCap:<\/strong><br><a rel=\"noreferrer noopener\" href=\"https:\/\/vcai.mpi-inf.mpg.de\/projects\/2020-cvpr-deepcap\/\" target=\"_blank\">Monocular Human Performance Capture Using Weak Supervision, CVPR 2020<\/a><\/li><li><strong>EventCap: <\/strong><br><a rel=\"noreferrer noopener\" href=\"https:\/\/vcai.mpi-inf.mpg.de\/projects\/2020-cvpr-eventcap\/\" target=\"_blank\">Monocular 3D Capture of High-Speed Human Motions using an Event Camera, CVPR 2020<\/a><\/li><li><strong>HandCap: <\/strong><br><a rel=\"noreferrer noopener\" href=\"https:\/\/vcai.mpi-inf.mpg.de\/projects\/2020-cvpr-hands\/\" target=\"_blank\">Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data, CVPR 2020<\/a> <\/li><li><strong>StyleRig: <\/strong><br><a rel=\"noreferrer noopener\" href=\"http:\/\/gvv.mpi-inf.mpg.de\/projects\/StyleRig\/\" target=\"_blank\">http:\/\/gvv.mpi-inf.mpg.de\/projects\/StyleRig\/<\/a> <\/li><li><strong>XNect: <\/strong><br><a rel=\"noreferrer noopener\" href=\"https:\/\/vcai.mpi-inf.mpg.de\/projects\/XNect\/\" target=\"_blank\">Real-time Multi-Person 3D Motion Capture with a Single RGB Camera, SIGGRAPH 2020<\/a> <\/li><li><strong>LiveCap: <\/strong><br><a rel=\"noreferrer noopener\" href=\"https:\/\/vcai.mpi-inf.mpg.de\/projects\/LiveCapV2\/\" target=\"_blank\">Real-time Human Performance Capture from Monocular Video, ToG 2019<\/a> <\/li><li><strong>MonoPerfCap: <\/strong><br><a rel=\"noreferrer noopener\" href=\"https:\/\/vcai.mpi-inf.mpg.de\/projects\/wxu\/MonoPerfCap\/\" target=\"_blank\">Human Performance Capture from Monocular Video, TOG 2018<\/a><\/li><\/ul>\n\n\n\n<h3 class=\"has-medium-font-size wp-block-heading\" id=\"\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\"><strong>\u25a0\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9<\/strong><\/h3>\n\n\n\n<p><a href=\"https:\/\/arithmer.blog\/wp-content\/uploads\/2022\/02\/07_\u753b\u50cf\u304b\u3089\u306e3D\u4eba\u4f53\u30e2\u30c7\u30ea\u30f3\u30b0.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">\u753b\u50cf\u304b\u3089\u306e3D\u4eba\u4f53\u30e2\u30c7\u30ea\u30f3\u30b0.pdf<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u672c\u8cc7\u6599\u306f2020\u5e747\u67089\u65e5\u306b\u793e\u5185\u5171\u6709\u8cc7\u6599\u3068\u3057\u3066\u5c55\u958b\u3057\u3066\u3044\u305f\u3082\u306e\u3092WEB\u30da\u30fc\u30b8\u5411\u3051\u306b\u30ea\u30cb\u30e5\u30fc\u30a2\u30eb\u3057\u305f\u5185\u5bb9\u306b\u306a\u308a\u307e\u3059\u3002 \u25a0Outline Introduction Definition of the problem Chal &#8230; <\/p>\n","protected":false},"author":3,"featured_media":989,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[13],"tags":[20,38,35,37,24,45,36],"_links":{"self":[{"href":"https:\/\/arithmer.blog\/index.php?rest_route=\/wp\/v2\/posts\/1000"}],"collection":[{"href":"https:\/\/arithmer.blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/arithmer.blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/arithmer.blog\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/arithmer.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1000"}],"version-history":[{"count":4,"href":"https:\/\/arithmer.blog\/index.php?rest_route=\/wp\/v2\/posts\/1000\/revisions"}],"predecessor-version":[{"id":1007,"href":"https:\/\/arithmer.blog\/index.php?rest_route=\/wp\/v2\/posts\/1000\/revisions\/1007"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/arithmer.blog\/index.php?rest_route=\/wp\/v2\/media\/989"}],"wp:attachment":[{"href":"https:\/\/arithmer.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1000"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/arithmer.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1000"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/arithmer.blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1000"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}