?

Log in

No account? Create an account

Previous Entry | Next Entry

A socialist blogging platform

While the untimely demise of LiveJournal is not likely in the immediate future, yesterday's furor over Sup's downsizing made me think, and what I thunk is this: the (technological) problem with LiveJournal is its single-point-of-failure capitalistic architecture. If LiveJournal isn't sustainable financially, it will disappear. Sure, someone will try to save it, and there will be attempts at organized moves to other weblog platforms, but any move big enough to maintain the LJ community's cohesiveness (which is what gives LJ its real value) will result in the same thing: if the chosen platform isn't profitable enough, it'll collapse.

On top of that, no other blogging platform of a reasonable size has the same community-enabling features as LJ. Blogging on Blogger or Wordpress or Typepad just doesn't feel the same; it's like posting in a vacuum, where people might stumble across your posts if you're lucky. Blogging on LJ feels like posting in a busy marketplace, where even though you don't know most people, there's a sense of life.

If LJ fails, we don't need to move to another blogging website. Even the fledgling Dreamwidth, supposedly the great white hope for when LJ goes under, isn't addressing the problems involved with having such a huge userbase. What we need is a blogging community platform that doesn't require profitability to sustain itself. Instead of users trading money (or ad impressions) for the use of Sup's server space and bandwidth, why not each user give storage space and bandwidth according to his ability, and use according to his needs?

The advantage of peer-to-peer architecture is the fact that it doesn't need a massive central pool of resources. There's no need for a database server farm or a web server farm or a massive amount of bandwidth or power; the only things needed are 1) a solid, secure piece of software, and 2) a critical mass of users. The whole point of this exercise is that we've already got a serious critical mass (which is the difficulty for traditional architectures), so all we need is to design and build a good p2p blogging platform (and then convince most of LJ to move to it in the event of a sudden LJ failure -- not a simple task, I know).

The basic system is fairly straightforward: You run a client, you write posts, you connect to other people and download their posts to your system and upload your posts to their system, everybody gets everybody else's posts, the end.

But if LJ has 3 million active users and they each write a 1kb post each week and each post gets a half-dozen 256-byte comments, that's 7.3gb of text for everybody to exchange each week, which is obviously less than optimal. So instead of downloading everything you get your hands on, you instead only download your friends' entries and comments and your friends' friends' entries and comments, and when a post hasn't gotten any new comments or updates in a month, you compress it and archive it to save space.

What about private posts? You throw PGP/GPG into the mix. When you write a private post locked to a group of 12 friends, you create 12 copies of it, each one encrypted with a different friend's public key, so only that friend can decrypt it. When you send out a query for your friends' posts, you're only given an encrypted post if it has your name (and public key) on its envelope. There's a scaling issue here, but it's a start.

Now what about communities? Private communities? Moderated communities? These are all totally solvable problems within a community-powered, cryptographically strong setup. A little imagination can come up with a solution to any of them.

The next big hurdle is encouraging adoption. Nobody likes to have to run a separate piece of software, so why not take the AllPeers source as a starting point and make this a browser extension? They've done a lot of the hard work: sockets, storage, communication, stuff like that. We'd just have to build on top of it to turn it into the platform we want. And we're already working within a web browser, so by and large the interface could be practically the same as LJ currently (for what that's worth).

As with most things, the only thing required to get this off the ground is a giant pile of effort. But with a userbase of millions and the potential threat of the sudden dissolution of the LJ community fresh in our minds, maybe it's something we can actually make happen.

Comments

( 18 comments — Leave a comment )
joenotcharles
Jan. 7th, 2009 04:55 pm (UTC)
Congratulations! You've invented Usenet!
joenotcharles
Jan. 7th, 2009 05:00 pm (UTC)
I mean, not really, but seriously - "you only download your friends' posts" = "you download the headers first and then only download the content if the From address is on your whitelist". Comments on a post is just threading. PGP/GPG signed posts can be done on Usenet, there's just no good interface. The only big difference is that everything is grouped into "topic groups" (like LJ communities) instead of groups owned by a person.

So, make an interface where it's easy to make your own alt.yourGUIDhere group, and add a nice interface to signing and encrypting posts, and you're most of the way done. (Ok, anyone can post to alt.yourGUIDhere, not just the person who created it, but that can be solved by setting the downloader to ignore as spam anything that's not signed by the owner of GUID, or a reply to a thread signed with that key.)
rfreebern
Jan. 7th, 2009 05:13 pm (UTC)
Sure, but USENET still requires central server resources, which is the main thing I'm trying to get away from here. I know the USENET infrastructure already exists and replication is taken care of, but it's also losing popularity quickly and, again, isn't in the users' hands.

It seems like my option is "build a client that encompasses and manages the entire platform", and yours is "build an interface to an existing platform to make it work the way you want a new platform to work" and that doesn't sound nearly as solid a solution.
joenotcharles
Jan. 7th, 2009 05:41 pm (UTC)
No it doesn't. Traditionally it's implemented where your ISP or university department or whatever runs a server, but running something like leafnode on your desktop works just as well.
joenotcharles
Jan. 7th, 2009 05:47 pm (UTC)
Also, I'm not actually advocating using Usenet as-is, I'm just saying that this is in many ways a return to the older architecture which got dumped when everyone jumped on the web.
rfreebern
Jan. 7th, 2009 06:01 pm (UTC)
Sure, I get that, and I agree that attempting to use the web for everything isn't the best decision. I'd like to see a move toward a p2p type architecture for many things, not just weblog communities, since I think the web's one-to-many delivery setup is inherently weak.
radiotelescope
Jan. 7th, 2009 06:06 pm (UTC)
Hire Nate Cull and start pounding his ideas into functional shape. :)
rfreebern
Jan. 7th, 2009 07:21 pm (UTC)
His ideas are fascinating. I don't understand them well enough to know how they apply to this sort of stuff, but I'm not surprised that they do. The Global Lisp Machine idea is similar, and intriguing.
snej
Jan. 8th, 2009 10:20 pm (UTC)
Yup, Usenet was definitely a classic peer-to-peer architecture, since they didn't have any other choice back in 1979 or whenever: those schools weren't lucky enough to have ARPAnet nodes, so they had their mainframes dial each other up by modem to exchange data. Then a couple years later the BBS scene re-invented the same idea.

Unfortunately Usenet wasn't a particularly good P2P network. That system would never scale to the kind of traffic Usenet or LJ handles today. That's what the Gnutella people discovered when they re-invented it once again...
radiotelescope
Jan. 7th, 2009 06:04 pm (UTC)
Or put it this way: when Usenet first got started, the average Usenet participant (or at least a large share of them) knew how to run the software him/herself. This is what you are talking about (ignoring the particular use of "client" or "server" re Usenet). Centralizing resources was a convenience.

(This is just as true for HTTP. And email. Everything, really. Okay, maybe not DNS, but I only say that because DNS is a dark cave of mysteries to me.)

I think the underlying question is, can the community-building power of Livejournal be decentralized *at all*. It's not so important if *one person* can keep a personal resource running; maybe it takes ten people, or a Dreamwidth-sized gang, or a LJ-sized gang. As long as new resources can be merged into the system, and not depend on the original "seed" resource (not be vulnerable to its failure), then people will be able to organize to the required degree. SMTP and HTTP have this property.

I *believe* that the Dreamwidth people are thinking about this problem, from their context of evolving Livejournal code into something better. I do not know any details, however. It is worth asking them.
ydna
Jan. 7th, 2009 08:49 pm (UTC)
Multiple recipients can be in the same encrypted text. It does not grow the size too much. At least, less than several copies. Except now each recipient will know who the other recipients are.
rfreebern
Jan. 9th, 2009 01:37 am (UTC)
It's been a long time since I've done anything with cryptography. Thanks for reminding me of that. :)
ydna
Jan. 8th, 2009 10:01 pm (UTC)
rfreebern
Jan. 9th, 2009 01:42 am (UTC)
Huh, interesting. Maybe he and I can bounce some ideas off each other.
snej
Jan. 8th, 2009 10:26 pm (UTC)
Hi! ydna pointed me to your post. I hope you'll read mine.

I have actually partly implemented a P2P social network system like what you describe. Unfortunately I ran out of sabbatical time and went back to work before I could finish it, and I haven't had time to work on it since. And also, it doesn't currently have any blog/journal features; the application it was designed for is more of a real-time collaboration system. But I designed the architecture knowing that journaling would be important.

Unfortunately P2P protocols are really hard. And cross-platform client-side apps are hard to do well too; I decided to focus on the Mac and write a native Cocoa app since that's my platform of choice and I have no desire to learn Windows APIs or use one of those crippled cross-platform toolkits.

So the proposal in my post tries to be more minimalist: how can we take the existing technologies of blogs, FOAF, Atom and OpenID and package them in an easily-deployable web app that individual users can install on their shared web hosts, and form a social network like LJ? I think it's quite do-able.
rfreebern
Jan. 9th, 2009 01:58 am (UTC)
Yeah, I have some experience implementing p2p systems as well, although that's a million years in the past. I realize that it's tough, but I also don't think it's nearly insurmountable -- and I really would like to see a shift away from the web's current architecture.

It seems to me like having Big, Powerful Web Servers doing the work for the puny masses only makes sense when the masses' bandwidth and storage is limited. Nowadays personal computers (by and large) are more powerful and more roomy, and networks are faster, than most people require on a day-to-day basis. If our networked applications could take advantage of that space and speed, I think it'd make the internet as a whole more robust.

The reason I thought of developing the client as a Firefox extension is that it's automatically just as cross-platform as the browser itself and it's effectively a one-click install. (Plus, the AllPeers folks have done a lot of the hard work.) I think most people would balk at having to set something up on a web host, even if it's made as simple as your Google App Engine proposal. And then, you're left with the same old "what if I no longer have access to this?" problem that people are worried about with LJ now.
snej
Jan. 8th, 2009 10:32 pm (UTC)
Oh, also: I like your socialist/capitalist analogy, but I actually see it the other way around: centralized systems like LJ/Facebook/Vox/etc. are the socialist ones in that a single centralized 'government' runs all the services; while a distributed system forms a free market where users can choose where to host their server, what implementation of the code to run, and can modify it as they see fit. The downside of the 'capitalist' distributed system being that those who don't have the capability to run a node are out in the cold, whereas the socialist central services will take anyone who can operate a web browser.
rfreebern
Jan. 9th, 2009 02:09 am (UTC)
I admit I don't understand the vagaries of socialism vs. capitalism too well, but I guess I saw Sup/LJ as the concentration of power here (doling out permissions/storage/etc. based on the money given them, and totally subject to the whims of the economy), and a distributed blogging platform as more egalitarian -- anyone who wants to install the software is suddenly part of the ecosystem, it's (relatively) economic-trouble-proof, etc.

Interesting, though.
( 18 comments — Leave a comment )

Latest Month

February 2011
S M T W T F S
  12345
6789101112
13141516171819
20212223242526
2728     
Powered by LiveJournal.com