{"id":14279,"date":"2016-04-26T13:29:59","date_gmt":"2016-04-26T17:29:59","guid":{"rendered":"http:\/\/mjtsai.com\/blog\/?p=14279"},"modified":"2019-01-04T15:33:59","modified_gmt":"2019-01-04T20:33:59","slug":"dropboxs-project-infinite","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2016\/04\/26\/dropboxs-project-infinite\/","title":{"rendered":"Dropbox’s Project Infinite"},"content":{"rendered":"

Yesterday<\/a>, I wrote about the superiority of BitTorrent Sync’s Selective Sync over Dropbox’s. Today, Dropbox<\/a> gave a technology preview (via Mitchel Broussard<\/a>, Hacker News<\/a>):<\/p>\n

\n

Project Infinite will enable users to seamlessly and securely access all their Dropbox files from the desktop, regardless of how much space they have available on their hard drives. Everything in the company’s Dropbox that you’re given access to, whether it’s stored locally or in the cloud, will show up in Dropbox on your desktop. If it’s synced locally, you’ll see the familiar green checkmark, while everything else will have a new cloud icon.<\/p>\n<\/blockquote>\n

It’s not clear whether this will be limited to business customers.<\/p>\n

antoncohen<\/a>:<\/p>\n

This feature is implemented at a low level, and works on the command line.<\/p>\n

For example if you have a directory that is all stored in the cloud you can cd<\/tt> to it without any network delay, you can do ls -lh<\/tt> and see a list with real sizes without a delay (e.g., see that an ISO is 650 MB), and you can do du -sh<\/tt> and see that all the files are taking up zero space.<\/p>\n

If you open a file in that directory, it will open, even from command line, then do du -sh<\/tt> and see that that file is now taking up space, while all the others in the directory are not.<\/p><\/blockquote>\n\n

Dan Luu<\/a> (in 2015):<\/p>\n

Something that’s occasionally overlooked is that hardware performance also has profound implications for system design and architecture. […] Consider the latency of a disk seek (10ms) vs. the latency of a round-trip within the same datacenter (.5ms). The round-trip latency is so much lower than the seek time of a disk that we can disaggregate storage and distribute it anywhere in the datacenter without noticeable performance degradation, giving applications the appearance of having infinite disk space without any appreciable change in performance. This fact was behind the rise of distributed filesystems within the datacenter ten years ago, and various networked attached storage schemes long before.<\/p>\n

[…]<\/p>\n

However, while it’s easy to say that we should use disaggregated disk because the ratio of network latency to disk latency has changed, it’s not as easy as just taking any old system and throwing it on a fast network. If we take a 2005-era distributed filesystem or distributed database and throw it on top of a fast network, it won’t really take advantage of the network. That 2005 system is going to have assumptions like the idea that it’s fine for an operation to take 500ns, because how much can 500ns matter? But it matters a lot when your round-trip network latency is only few times more than that. The caching and other obvious wins if you have 1ms latency may not buy you much at 10us latency, and it may even cost you something.<\/p>\n

Latency hasn’t just gone down in the datacenter. Today, I get about 2ms to 3ms latency to Youtube. Youtube, Netflix, and a lot of other services put a very large number of boxes close to consumers to provide high-bandwidth low-latency connections. A side effect of this is that any company that owns one of these services has the capability of providing consumers with infinite disk that’s only slightly slower than normal disk.<\/p><\/blockquote>\n

Dan Luu<\/a>:<\/p>\n

We’ll see how well the implementation works, but when Dropbox was first released people said, “who cares, it’s just rsync”.<\/p><\/blockquote>","protected":false},"excerpt":{"rendered":"

Yesterday, I wrote about the superiority of BitTorrent Sync’s Selective Sync over Dropbox’s. Today, Dropbox gave a technology preview (via Mitchel Broussard, Hacker News): Project Infinite will enable users to seamlessly and securely access all their Dropbox files from the desktop, regardless of how much space they have available on their hard drives. Everything in […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"apple_news_api_created_at":"2019-01-04T20:34:02Z","apple_news_api_id":"ee524a8b-f4c1-48b7-9781-2f8250506483","apple_news_api_modified_at":"2019-01-04T20:34:04Z","apple_news_api_revision":"AAAAAAAAAAD\/\/\/\/\/\/\/\/\/\/w==","apple_news_api_share_url":"https:\/\/apple.news\/A7lJKi_TBSLeXgS-CUFBkgw","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_is_hidden":false,"apple_news_is_paid":false,"apple_news_is_preview":false,"apple_news_is_sponsored":false,"apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":"\"\"","apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"footnotes":""},"categories":[2],"tags":[47,30,32,1771,269],"class_list":["post-14279","post","type-post","status-publish","format-standard","hentry","category-technology","tag-dropbox","tag-mac","tag-macapp","tag-rsync","tag-syncing"],"apple_news_notices":[],"_links":{"self":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/14279","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/comments?post=14279"}],"version-history":[{"count":1,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/14279\/revisions"}],"predecessor-version":[{"id":14280,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/posts\/14279\/revisions\/14280"}],"wp:attachment":[{"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/media?parent=14279"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/categories?post=14279"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mjtsai.com\/blog\/wp-json\/wp\/v2\/tags?post=14279"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}