{"id":4240,"date":"2012-02-03T20:04:13","date_gmt":"2012-02-04T01:04:13","guid":{"rendered":"http:\/\/mjtsai.com\/blog\/?p=4240"},"modified":"2012-02-03T20:06:30","modified_gmt":"2012-02-04T01:06:30","slug":"git-at-facebook-scale","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2012\/02\/03\/git-at-facebook-scale\/","title":{"rendered":"Git at Facebook Scale"},"content":{"rendered":"<p><a href=\"http:\/\/thread.gmane.org\/gmane.comp.version-control.git\/189776\">Joshua Redstone<\/a> (via <a href=\"http:\/\/news.ycombinator.com\/item?id=3548824\">Hacker News<\/a>, which has some interesting comments):<\/p>\n<blockquote cite=\"http:\/\/thread.gmane.org\/gmane.comp.version-control.git\/189776\"><p>The test repo has 4 million commits, linear history and about 1.3 million\nfiles.  The size of the .git directory is about 15GB, and has been\nrepacked with 'git repack -a -d -f --max-pack-size=10g --depth=100\n--window=250'.  This repack took about 2 days on a beefy machine (I.e.,\nlots of ram and flash).  The size of the index file is 191 MB.<\/p><\/blockquote>\n<p><a href=\"http:\/\/thread.gmane.org\/gmane.comp.version-control.git\/189776\">Sam Vilain<\/a>:<\/p>\n<blockquote cite=\"http:\/\/thread.gmane.org\/gmane.comp.version-control.git\/189776\"><p>With the hard&#8211;working part of git on the other end of a network service, \nyou could back it by a re&#8211;implementation of git which is written to be \ndistributed in Hadoop.  There are at least two similar implementations \nof git that are like this: one for cassandra which was written by github \nas a research project, and Google's implementation on top of their \nBigTable\/GFS\/whatever.  As the git object storage model is write&#8211;only \nand content&#8211;addressed, it should git this kind of scaling well.<\/p><\/blockquote>","protected":false},"excerpt":{"rendered":"<p>Joshua Redstone (via Hacker News, which has some interesting comments): The test repo has 4 million commits, linear history and about 1.3 million files. The size of the .git directory is about 15GB, and has been repacked with 'git repack -a -d -f --max-pack-size=10g --depth=100 --window=250'. This repack took about 2 days on a beefy [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"apple_news_api_created_at":"","apple_news_api_id":"","apple_news_api_modified_at":"","apple_news_api_revision":"","apple_news_api_share_url":"","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_is_hidden":false,"apple_news_is_paid":false,"apple_news_is_preview":false,"apple_news_is_sponsored":false,"apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":"\"\"","apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"footnotes":""},"categories":[4],"tags":[],"class_list":["post-4240","post","type-post","status-publish","format-standard","hentry","category-programming-category"],"apple_news_notices":[],"_links":{"self":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/4240","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/comments?post=4240"}],"version-history":[{"count":5,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/4240\/revisions"}],"predecessor-version":[{"id":4245,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/4240\/revisions\/4245"}],"wp:attachment":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/media?parent=4240"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/categories?post=4240"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/tags?post=4240"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}