Google Engineers Behind Go/Golang Create New Global Namespace, Upspin

[TheRealFierce]

[QUOTE]Wouldn't it be nice if there was a universal and consistent way to give names to files stored on the Internet, so they were easy to find? A universal resource locator, if you like?

The problem is that URLs have been clunkified, so Upspin, an experimental project from some Google engineers, offers an easier model: identifying files to users and paths, and letting the creator set access privileges.

By the by, yes: Dropbox, Google, Apple, Microsoft, and World+Dog offer file-sharing – but Upspin's authors reckon file sharing that way is also clunky.

Most of all, however, it's the big names (Google included) that Upspin's engineers reckon are making it hard to share stuff. As they note in their documentation:

[B]“Photos that one uploads to Facebook or Twitter are managed by, and effectively belong to, those providers.

“Other than occasional workarounds using a URL, information given to these services becomes accessible only through those services. If one wants to post a Facebook picture on one’s Twitter feed, one does that by downloading the data from Facebook and then uploading it to Twitter. Shouldn’t it be possible to have the image flow directly from Facebook to Twitter?

“This 'information silo' model we have migrated to over the last few years makes sense for the service providers but penalises the users, those who create and should therefore be in charge of their data.”[/B]

The Google software engineers who put Upspin together (Andrew Gerrand, Eric Grosse, Rob Pike, Eduardo Pinheiro and Dave Presotto) add that users need more granular access control than “private” and “public”, plus better security and privacy controls.

Hence Upspin: not an application, but a framework to create something like a global namespace based on a user, identified by their e-mail address.

As explained at the project's [URL="https://upspin.googlesource.com/upspin/"]repository[/URL] (there's a mirror at [URL="https://github.com/upspin/upspin"]GitHub[/URL]), “it is not a file system, but a set of protocols and reference implementations that can be used to join things like file systems and other storage services to the name space.”[/QUOTE]

[QUOTE]The naming framework is ridiculously simple: it's based on filepaths that attach to the user, thus [email][e-mail][/email]/dir/file is what an Upspin service will evaluate to reach the file [email][e-mail][/email] shares.

There's no concept of a “local path” in the system – every file including local files, have to be fully qualified. And, in a borrowing from Unix, files can also be links (for example, to directories or other users' files).

Users grant access to their files with a file which, per-directory, identifies who has access. With no access file, [email][e-mail][/email]/dir/file is private to Ann; if Access exists in the directory, Ann can give others access to that directory with a simple:

read: [email][e-mail][/email], [email][e-mail][/email] [/QUOTE]

[url]https://www.theregister.co.uk/2017/02/22/upspin_debuts/[/url]

[QUOTE]Upspin is an experimental project to build a framework for naming and sharing files and other data securely, uniformly, and globally: a global name system of sorts.

It is not a file system, but a set of protocols and reference implementations that can be used to join things like file systems and other storage services to the name space.

Performance is not a primary goal. Uniformity and security are.

Upspin is not an official Google product.[/QUOTE]

[url]https://upspin.googlesource.com/upspin/[/url]

Very interesting stuff, a few of the people behind the project are former Bell Labs employees, which were responsible for things like Unix and Plan 9, who later went on to work at Google and develop the Go/Golang programming language.
Also as with most projects of their projects they have a mascot too, Augie

[t]https://upspin.io/images/augie-transparent.png[/t]

You can read the documentation here:
[url]https://upspin.io/[/url]

[Glitchvid]

Why not just go the whole way and make the whole layer distributed and hash based?
It's what [URL="https://ipfs.io/"]IPFS[/URL] does:
[media]https://www.youtube.com/watch?v=HUVmypx9HGI[/media]

[343N]

[QUOTE=glitchvid;51857532]Why not just go the whole way and make the whole layer distributed and hash based?
It's what [URL="https://ipfs.io/"]IPFS[/URL] does:
[media]https://www.youtube.com/watch?v=HUVmypx9HGI[/media][/QUOTE]

i understand if it's important that i watch it all, but, summary please?

[Lolcats]

google devs trying to solve a problem that doesn't exist. or could be solved better. their overview page is the most wordy block of pure black on white text i've seen, and is more of a proof of concept than anything else.

there's a reason most places have you upload files to their servers - so they can easily distribute it, control it, and monitor it. Or maybe I'm just being critical.

[QUOTE=343N;51857631]i understand if it's important that i watch it all, but, summary please?[/QUOTE]

IPFS (Interplanetary File System) is a distributed web kind of program. iirc the way it works is every server running ipfs software holds a little bit of the ipfs "block", that is, all the content that has been uploaded to IPFS. You can read a better condensed version on their webiste: [url]https://ipfs.io/#how[/url]

IPFS is pretty much what i was referencing when I was talking about "be solved better".

[Glitchvid]

[QUOTE=343N;51857631]i understand if it's important that i watch it all, but, summary please?[/QUOTE]

Basically it's a replacement for HTTP, where instead of files having a hostname then URL to the file, everything is linked by a hash of the file. This means it's entirely p2p, and has built-in perminance (A file can always be found as long as it exists somewhere on the network).

[mastersrp]

[QUOTE=glitchvid;51857899]Basically it's a replacement for HTTP, where instead of files having a hostname then URL to the file, everything is linked by a hash of the file. This means it's entirely p2p, and has built-in perminance (A file can always be found as long as it exists somewhere on the network).[/QUOTE]

I don't recall exactly how it works, but it seems to have the same functional issues as Freenet does: If a file or parts of a file stop being popular long enough, then they may fall out of the network.
That means that data is only available on the network, so long as someone is making it available AND someone else is interested in it, due to space constraints (which is actually a real issue, even for something like IPFS and Freenet. Users do not have unlimited space, so the total amount of space used by files + redundancy blocks is constrained).

[Tamschi]

[QUOTE=Lolcats;51857826][...]

IPFS (Interplanetary File System) is a distributed web kind of program. iirc the way it works is every server running ipfs software holds a little bit of the ipfs "block", that is, all the content that has been uploaded to IPFS. You can read a better condensed version on their webiste: [url]https://ipfs.io/#how[/url]

IPFS is pretty much what i was referencing when I was talking about "be solved better".[/QUOTE]

It seems to solve a very different problem though, unless there are easy ways to control access and delete items uploaded to IPFS.

I'm not saying that Upspin is necessarily particularly good, but IPFS is probably a lot worse for this particular problem.

[Lolcats]

[QUOTE=Tamschi;51857941]It seems to solve a very different problem though, unless there are easy ways to control access and delete items uploaded to IPFS.

I'm not saying that Upspin is necessarily particularly good, but IPFS is probably a lot worse for this particular problem.[/QUOTE]

If people want restricted access to what they share, they probably wouldn't be sharing it on Facebook/Twitter in the first place. And if they're looking to share something privately computer to computer, well, sure there's nothing stopping me from taking a link you send me and sending it to someone else, but there's also nothing stopping me from taking the picture you send me from Upspin (downloading it or screen capturing it), and simply rehosting it elsewhere.

And there are plenty of file sharing services that encrypt your uploads. Upspin is pretty much a more hands on approach Dropbox:
[QUOTE=Upspin]
By default, nothing is shared. When a user is added to Upspin, all data created by that user is invisible to everyone else.

If the user wishes to share a directory (the unit at which sharing is defined), she adds a file called Access to that directory. In that file she describes, using a simple textual format, the rights she wishes to grant and the users she wishes to grant them to. For instance, an Access file that contains

read: [email][e-mail][/email], [email][e-mail][/email]
[/QUOTE]

Then again I'm just glossing their overview sheet. I could be missing a key point.

[TheRealFierce]

[QUOTE=Lolcats;51858033]If people want restricted access to what they share, they probably wouldn't be sharing it on Facebook/Twitter in the first place. And if they're looking to share something privately computer to computer, well, sure there's nothing stopping me from taking a link you send me and sending it to someone else, but there's also nothing stopping me from taking the picture you send me from Upspin (downloading it or screen capturing it), and simply rehosting it elsewhere.

And there are plenty of file sharing services that encrypt your uploads. Upspin is pretty much a more hands on approach Dropbox:
[/QUOTE]

Upspin in still a project very much in its infancy which includes it's documentation, which seems quite vague. I'm not entirely sure what the benefits of the project are. But given the team behind it, it seems it might have a promising future, probably.

In addition the privacy elements are not to protect the file from being only viewed by the recipients of the file, that's dumb and probably impossible. It's supposed to protect the file from being only accessed by the recipients of the file and no man-in-the-middle. If you decide to post something publicly that was meant to be private between you and the sender, that's not a flaw in privacy, that's a flaw in trust.

Also it's different in regards to Dropbox. It's true Dropbox encrypts your files, but only with a single key. This means that if someone were to somehow have access to all the files stored in Dropbox's servers they wouldn't be able to view the contents of the files as they would be encrypted. [I]But[/I] if they happen to have the key as well, they would have full access to all the files. This also means Dropbox themselves have access to the files stored on their server, should they choose to decrypt them. From what I can tell Upspin requires files to be encrypted when they are stored in the server. Each user is given their own key which would decrypt their files, and if they choose to share the file with someone else, their keys would also be able to decrypt the file.

In that regard it would seem that Upspin has more in common with the [URL="https://keybase.io/docs/kbfs"]kbfs[/URL] than it does with Dropbox. Although it seems that Upspin's goals has more to do with integrating itself into websites and decrease reliance on having to upload files to one or more websites central servers, as well as unifying all the user's files from phones, cameras to one place. But then again I'm sure there are other services that does those things just as well or even better. Again the documentation on it seems pretty vague.

Another thing to mention is as it is in its current state, it's not useable by it's intended users at all as mentioned on their website. Fumbling around with keys, username emails and whatnot is far from what most would consider average user friendly.