Wired Donations

I received several requests from users who would like to donate money to support the Wired project. I added a donation button (PayPal) on the Wired website.

These donations are intended to support and encourage the development of the Wired project in the open-source way.

Thank you for supporting this awesome project.

Best regards,

RW

Dear Wiredians

A few months ago, I decided to continue the Wired 2.0 project Morris gave up (see this post). I currently get good results regarding bug fixing, performance tuning and stability. Wired Server (both UNIX and OSX), Wired Client and Wire received several enhancements and regular updates. More, we now have a piece of website: http://www.read-write.fr/wired

Until now, Wired has remained quite a secret, almost elitist community. It is good, but also bad: Wired lacks invested contributors because its lack of visibility. I truly think that Wired deserves to be more popular, especially in the UNIX world.

So, it is time to enlarge our community. As you know, Wired is licensed as open-source, and such a project needs a strong user community to live. Unfortunately, I am still the only developer working on the source code. Many of us think that only one coder cannot manage the entire project. Everybody is welcome to help, and if you are interested in contributing, we are looking for the following skills:

  • Coders: ANSI-C / Objective-C / Shell
  • Testers: UNIX / OSX
  • Writers: documentation / website / community / etc.

However, I know open-source a bit, and obviously nothing can run properly without a minimum of ethics. The following points are few rules to keep in mind:

  1. Wired is for fun. We use Wired to share good time first of all, we are not here to compete.
  2. Open-source is built on common sense, free knowledge and generosity. Do your best to share these qualities as much as possible.
  3. Learn by yourself as much as possible. Be helpful as much as possible.

I said it, now I also have some good news to reveal. As I’m working on the Wired project, I obviously have a lot of ideas and I received several enhancement requests directly related to the network protocol. Several features are retained to be implemented (if possible) in future versions:

  • Shared boards: give the ability to a board and its content to be shared between several servers. The best way to accomplish this is to use trackers which now support secure connections. We can imagine that a board is shared to a tracker, and that every server registered to this tracker receive board updates.

  • Offline messaging: provide a support for secure offline messaging. It is not easy to accomplish because Wired is based on strict security constraints, like the hidden nature of the login field.

  • iPhone client: I also worked on the iPhone client, with success. The project is on standby because I’m primarily working on existing server/client software to have a running environment. We definitely need more coders…

I hope to see you on our public server using the following clients and connection parameters:

Best regards,

RW

My guide for SSHFS on OS X

Introduction

The SSH (Secure Shell) protocol provides multiple extensions like SCP (Secure Copy), SFTP (Secure File Transfer Protocol), or SSHFS (Secure Shell File System). This guide shows how to use SSHFS in order to mount a remote directory as a volume on your file system.

Every serious FTP client software provides SCP/SFTP protocols implementation, but why to use third-party software when it is all about UNIX? I tried several apps which handle SSHFS and provide mount support, but I encountered many issues mainly like the instability of the Finder.

The way I found to get this working (stable) with an easy setup is using fuse4x, sshfs and few Applescript to automatize the process.

Requirements

This guide assumes that you already have an access on a running SSH server, configured with key exchange authentication (passwords are bad because transferred in plain text). Instructions below are available for Mac OS X 10.6 and higher.

Installation

So, to run SSHFS on Mac OS X, you must install several components:

fuse4x

fuse4x is an implementation of FUSE (File System in Userspace) for Mac OS X. Download it here and install the package.

sshfs

sshfs is the file system client based on SSH protocol we will use to mount a remote directory. There are several ways to install sshfs on your system:

  • Manually, using the Terminal (recommended):

    1. Download sshfs
    2. Untar: tar -xzvf sshfs-fuse-2.3.tar.gz
    3. Change directory: cd sshfs-fuse-2.3
    4. Configure: ./configure
    5. Compile: make
    6. Install: sudo make install
  • Using Macport:

    • sudo port install sshfs
  • Using Homebrew:

    • brew install fuse4x sshfs

I recommend the manual installation because it works perfectly for sshfs 2.3 version. Package managers like Macports and Homebrew are as sexy as confused and yet, why to use third-party software when it is all about UNIX?

To test if your installation successfully worked, open a new shell, and try the following command: sshfs -h. You should see the sshfs help message. If not, have a look to the References section at the bottom of this article, maybe you can found a little help.

How-to use

To mount a remote directory from a SSH host you need a dedicated local directory. So, on your local computer, open a Terminal and type:

mkdir ~/RemoteHome

Now, you will use sshfs to mount your remote directory to the newly local directory:

sshfs -p 22 user@example.net:/home/user/ ~/RemoteHome -oauto_cache,reconnect,defer_permissions,negative_vncache,volname=RemoteHome

If you go to ~/RemoteHome, you should see the content of /home/user/ at example.net.

To unmout the volume, use:

umount ~/RemoteHome

Tuning

By default, Mac OS X also writes « .DS_Store » files into your remote directories. It exists two ways to get rid of this issue:

  • Using the noappledouble option (specified remote volume):

    • sshfs -p 22 user@example.net:/home/user/ ~/RemoteHome -oauto_cache,reconnect,defer_permissions,noappledouble,negative_vncache,volname=RemoteHome
  • Telling to the system to not write these such files (for every remote volumes…):

    • defaults write com.apple.desktopservices DSDontWriteNetworkStores true

To remove previously created « .DS_Store » files, use the following command:

find /path/to/share -name .DS_Store -exec rm {} \;

Automatize

Now, I would like to use SSHFS without typing any command line in the Terminal. I made a little script, using Applescript, to automatize the mounting of several directories in one click.

To use the script: 1. Open « Script Editor.app » and paste the script below 2. Adjust SSH parameters: sshHostname, sshPort and sshLogin 3. Adjust SSHFS parameters: remotePaths, localPaths and volumeNames 4. Save the script as an application 5. Run the newly created application!

The script here:

-- SSH parameters 
 set sshHostname to "example.net" 
 set sshPort to 22 
 set sshLogin to "user"

-- SSHFS parameters
set remotePaths to {"/home/user/Documents", "/home/user/Downloads"}
set localPaths to {"~/Desktop/Documents", "~/Desktop/Downloads"}
set volumeNames to {"Documents", "Downloads"}

repeat with x from 1 to count of remotePaths
    set remotePath to item x of remotePaths
    set localPath to item x of localPaths
    set volumeName to item x of volumeNames

    -- Try to unmount
    UnmountVolume(localPath)

    -- Check local directory path
    CheckVolumeDirectory(localPath)

    -- Try to mount
    MountVolume(sshHostname, sshPort, sshLogin, sshPassword, localPath, remotePath, volumeName)
end repeat

-- Function: Create directory if not exists
on CheckVolumeDirectory(aPath)
    tell application "System Events"
        if exists file aPath then
        else
            try
                do shell script "mkdir " & aPath
            end try
        end if
    end tell
end CheckVolumeDirectory

-- Function: Unmout volume at path
on UnmountVolume(aPath)
    try
        do shell script "/sbin/umount " & aPath
        log "Unmount SSHFS Volume: " & aPath
    end try
end UnmountVolume

-- Function: Mout volume at path
on MountVolume(sshHostname, sshPort, sshLogin, sshPassword, localPath, remotePath, volumeName)
    try
        set sshURL to sshLogin & "@" & sshHostname

        do shell script "/usr/local/bin/sshfs -p" & sshPort & " " & sshURL & ":" & remotePath & " " & localPath & " -oauto_cache,reconnect,defer_permissions,noappledouble,negative_vncache,volname=" & volumeName
        log "Mount SSHFS Volume: " & localPath
    end try
end MountVolume

References

Wired disaster

I have decided to take up the development of Wired myself. The project is called Wired, and is an open-source project written by Axel Andersson. His nickname on the project is Morris and he releases the software as Zanka Software. He created the project in 2003 with the goal of providing a BBS-style (Bulletin Board Software) server/client system for Max and UNIX platforms.

Mr. Andersson has been working for Skype for two years now and during this time Wired has been inactive. People are still using the Wired application but both the software and the community are far from what they should be. Before letting the project go to work full time for Skype, Morris started developing a 2.0 protocol which isn’t backwards compatible with the protocol from 1.1. 2.0 offered many new features but only some of the community have migrated.

When Morris left for Skype, he didn’t release latest available binaries. So, users are running the 2.0b51 version of the protocol, and source code repository hosts 2.0b53. Problem is that 2.0b51 and 2.0b53 versions of the protocol are not compliant, which is not a good start.

The result is that the community is split between users using different, incompatible protocols which leads to compatibility disasters. Restarting the project would be too difficult and would require massive amounts of work; there are no other existing pieces of software like Wired that are specifically engineered for the OS X platform.

Current state of the project

The original Wired software suite was composed of :

  • libwired : Core engine for the Wired network suite (written in C). It contains collections and other data structures, and portable abstractions for many OS services, like threads, sockets, files, etc.
  • wired : UNIX-based Wired server
  • wire : UNIX-based Wired client in console mode
  • trackerd : Tracker deamon to register Wired servers
  • Wired Client : Mac OS X client application
  • Wired Server : Mac OS X server preferences panel

It is also important to note that there are three other components that were designed specifically for Mac OS X:

  • WiredFoundation.framework : A set of Objective-C helpers for Wired support
  • WiredNetworking.framework : Objective-C encapsulation of the Wired network protocol
  • WiredAppKit.framework : A set of extensions and classes build against AppKit framework

This is all very good, and actually took quite a lot of work! There is clearly something missing from the list, Wired Mobile. Mobile software and development is becoming very prominent today, but it was not the same just a few years ago when Morris began to work on the first version of Wired. In Morris’ latests commits to the repository, he created static library targets against the Foundation framework and built configurations to help write Wired based software for the iOS platform, but there have been no efforts since then.

The Xcode 4 migration was a disaster for the Wired project. Morris used an Xcode built configuration system and shell scripts to integrate the compilation of the libwired library into the standard building process of a Cocoa application, without breaking the UNIX-based alternative. This was all done in order to allow multiple SDK targets in a single Xcode 3.x project, which is useless, as Xcode 4.x natively supports multiple SDK targets.

Morris designed Wired with longevity and portability in his mind, and I love his way of thinking about open source software development. Unfortunately, as Morris said, all of his work on Wired 2.0 is entirely undocumented. As you can see, version 1.1 has documentation.

Of course, other developers worked on Wired and/or it’s alternatives:

  • AlienWire : entirely forgotten today, it was an attempt to make a Win32 compliant client
  • Qwired : discontinued, it was an attempt to make a Qt-based cross platform version of the protocol
  • ssWired : totally in development currently, it is an attempt to make a cross-platform client for Wired
  • iWired : I plan to make my own client

NOTE : The official source repository is currently dead, but you can find a clean mirror here. I also host a mirror at GitHub.

Restarting the project

I have been dreaming of making my own BBS style software for Macs for a long time. It is Wired which motivated me to begin computer programming; we tweaked chat bots (wirebot, Chantal-BOT, then Plorkian-BOT) for Wired 1.1, which were written in C or Perl by members of the community. I then decided to restart the development of Wired as best as possible. I had already written some things like Wired Bot that were compliant with the Wired protocol, so it is safe to say that I was already familiar with it before I started.

Initially the most difficult part was to fix many issues that arose in the time that the project had previously been inactive. Apple platforms had changed a lot: Xcode 4, Objective-C 2.0, LLVM, loss of support for PPC architecture and mainly the introduction of the iOS platform.

Restarting a project that was started by another developer isn’t easy. Every developer has their own way of organizing and composing a project. Morris was the only developer of wired and you can tell when you read the code, which does not make work easy. Hopefully Morris did a smart and clean job with his code :-)

Why 2.0 if it’s a mess ?

The work already accomplished for 2.0 is much more than you may think, and it’s beautiful! If you read the Wired 1.1 documentation and specifications, you will learn that initially Wired was originally a very simple text-based message protocol. When you send a chat message, your client sends an encrypted simple string like « SAY chatID / Hello! \n » to the server. This was very light and easy to understand, but it’s disadvantage is that it isn’t scalable at all. Maintaining many different versions of this sort of protocol often creates inconsistencies in the implementation. Today, we see the massive use of JSON or XML as data structures in web architectures, which demonstrates the smoothness of this kind of solution.

Wired 2.0 introduces an XML-based specification called « p7″ that resolves the scalability issues of the protocol. It also introduces a Sqlite3 database for the server side which will make the system more open for future implementations. Morris also implemented new features, like the remote administration and board system. All of these additions make Wired a very strong, modern, scalable, and user-friend BBS solution. Even if the 2.0 specifications are not perfect speaking strictly from a functionally point of view, restarting the project from the 1.1 specifications is too cumbersome.

As a Cocoa developer, I am mainly able to maintain Objective-C code. The Wired library and server are written in pure C. It is a good opportunity to familiarize myself with this kind of project. I obviously know C, but not as well as Morris. Maybe it has to do with with the generation gap: while I learned mainly objected-oriented programming, Morris likely learned older programming styles & languages :-)

Objectives

First, we need to redistribute original binaries for the 2.0 server and client versions to pass the 2.0b51 vs. 2.0b53 gap, and to resync distributed sources and binaries. A new version will also help the community with the Wired 2.0 migration.

Next, we will have to fix the current 2.0 bugs (« we, because I can’t do this alone). The remaining issues may be difficult, which would explain why Morris didn’t do it already himself. After we finish, we can start to think of new features (I already have a very long list).

I also plan to develop Wired client apps for both OSX and iOS with a modern and clean UI. I’ve already started. I will publish a few screenshots when I have enough to show. I don’t know if apps distributed on the store will be free or not, under an open-source or a commercial license. Don’t worry, if it you need to buy it, it’s still going to be really cheap! Just enough to maintain project resources.

Finally, I think we need to document the project as best as we can. Wired must stay licensed as open-source and the documentation is the key to it’s success. Users and developers must be well informed, and will benefit greatly from better communication.

Already done

I’m currently writing a few classes to facilitate the creation of a Wired API on both OSX and iOS platforms. Two new frameworks (WiredData and WiredKit) will provide an Objective-C block-based API to manage Wired connections and a Core-Data layer to handle and persist Wired data easily. I’m also regrouping a few other sources to make a WiredAppKit, which is designed for iOS using the UIKit framework instead of AppKIt.

I hope that the community is ready to involve itself in the new Wired. I will need testers, support, and goodwill. The project is open to any useful contributions. Network programmers with good C skills are welcome. It will also be necessary to think about app translations in the near future. Gradually, I will host and share as many resources as possible like beta versions, documentation, articles, sources, etc.

It is time for me to dig into the awesome piece of software that Wired is. Stay in touch here; I will post news about the subject.

Cocoa Distributed Object (part. 1)

Over the past twenty years, programming languages have evolved towards the object paradigm. Environments have been adapted in terms of data persistence and allocation of memory and processor resources. But, faced with severe constraints posed by infrastructures, network oriented programming interfaces (APIs) have suffered to run at a too low level, and to create a logical bottleneck.

This paper aims to present an introduction to distributed programming through Objective-C language and Cocoa API. It is also, and formost, an introduction to network programming paradigms, with some tips to understand it with revelance.

Network programming

Network programming is complex because it tends to be interactive, just as it is sometimes hazardous –related to overload management, fault resistance, and security issues. In addition, the Open Systems Interconnection (OSI) model constrains to serialize data from the application layer to the physical layer, to achieve the transmission between peers. This process is very onerous, in terms of programming and source of errors.

As a result, the network APIs can be sumarized to the Socket API. It is commonly supplied with the operating system, and provides access to capabilities for exploiting the network founctionality of the computer. Sockets can easily create a stream between two remote computers to transfert data. Data exchange is performed at low-level, byte by byte, in a synchronous system confined by blocking functions (read and write). All this stuff is very far from object paradigm, and the serialization of objects (without breaking the object graph) produces an huge loss of time and energy.

Distributed objects

Fortunately, great people have solved the problem and found a very neat solution. The distributed objects concept appeared in the 90s. Several distributed architectures have flourished, based on remote procedure calls (RPC). It is intresting to note that one of the first serious implementation of this concept was created around Apple, by NeXT Computer Inc. with an API named Portable Distributed Objects. Objective-C language and Cocoa API inherit from it to provide an easy to use network programming interface, very powerfull compared to other implementations. There are other environments, such CORBA (Common Object Request Broker Architecture), which is an OMG (Object Management Group) standard that would like to provide a strong interoperability between various kind of implementations. And, if you know Java a little, you should know RMI (Remote Method Invocation).

Interactions

There is still a subtlety to not dismiss. It is obvious that a such programming interface also use Sockets to access to the network functionalities. A whole machinery thus facilitates the objects data descent into hell, and the API serializes objects and manages streams, transparently to the programmer. In counterpart, the distributed object system also suffers of constraints related to the Socket API.

The Internet Protocol (IP) encapsulates two main transport protocols : TCP (Transmission Control Protocol) and UDP (User Datagram Protocol), which are respectively reliable and unreliable protocols. TCP is reliable because it provides the control of the transmission (CRC, congestion, etc), which is possible by keeping the connection alive : it is a synchronous communication mode (connected mode). UDP doesn’t provides this kind of feature; it just sends/receives data from point to point, without controling integrity and flow : it is asynchronous mode (not connected mode).

So, distributed objects support two different kind of messages. First are one way messages, for which we will not wait for a return value : there are asynchronous. Others are round-trip messages, which provides a mechanism to return a value, in a synchronous way. Plainly, messages specification are abstracted by object methods in specific programming interfaces. In Objective-C, the Runtime uses protocols to define messages. Native methods are considered synchronous, and then, to declare an asynchronous message, you must use oneway syntax keyword. It’s obvious that a oneway method must also return a void value.

All this theory is needed to build a good communication protocol between multiple peers. The example below describe a situation where the use of asynchronous message instead of synchronous message, changes the game. More, we will see that is really simple to create messages specification using Objective-C protocols; perhaps too simple in fact, and that is why it is very imporant to keep the concept in mind.

Practice

We will consider a client entity, that connects to a server entity to send a simple “backup” message without parameters or return value (using void). Once the server received the message, it will launch a five seconds operation. We will see that according the used communication mode (synchronous or asynchronous), the client entity behavior is totally different.

Create the protocol

The protocol used here is made of a single message : backup.

@protocol BackupServerProtocol
- (void)backup;
@end;

Pay attention that a single message doesn’t necessarily imply a single interaction. If you don’t declare a method as oneway, the message involves two network intercations :

Firt, the client entity sends the message to the server entity (which do the work…) Next, the server entity sends an acknowledgement message to say that he received the sent message and work is done (althrough it has no value to return) This behavior is consistent to the TCP behavior. In fact, when you use synchronous methods, you use TCP transport layer, and then Sockets control the transmission to confirm the smooth running of your operation.

Create the server

Now we have a protocol, we can write the server object which will implement it. The protocol allows to abstract object type to create what is called a remote interface. This feature hide low-level operations which do the work (Sockets), and a NSConnection object handles remote object communications.

@interface BackupServer : NSObject   { }
@end

@implementation BackupServer
- (void) backup {
    NSLog(@"backuping : ");
    for(int i = 0; i < 5; i++) {
        NSLog(@".");
        [NSThread sleepForTimeInterval:1.0];
    }
}
@end

Run the server:

After, we must instanciate a server object, which makes it available from outside the program. The NSConnection object will handle the distributed object instance and will store its reference to a global register to ensure that it is traceable. The object is stored with a key (here the used key is “server”), used to recover it; this register is called a name server and responds to the NSPortNameServer class. Apple call this process “vending of an object”.

int main (int argc, const char * argv[]) {
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];

    NSConnection *theConnection;
    BackupServer * backupServer = [[BackupServer alloc] init];

    theConnection = [NSConnection new];
    [theConnection setRootObject: backupServer];

    if ([theConnection registerName:@"server"] == NO) {
        NSLog(@"Impossible to vend this object.");
    } else {
        NSLog(@"Object vended.");
    }

    [[NSRunLoop currentRunLoop] run];
    [pool drain];
    return 0;
}

NOTE: In this main function, I took care to run the current loop which is responsible of vended object input. This is not needed for a GUI based program, because NSApplication runs its own loop.

The client program To exploit this backup service, we need a client program. The client program will be bound with the remote interface registered as “server” to invoke the backup method.

int main (int argc, const char * argv[]) {
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
    NSConnection *theConnection;
    id backupServer;

    theConnection = [NSConnection connectionWithRegisteredName:@"server" host:nil];
    backupServer = [[theConnection rootProxy] retain];
    [backupServer setProtocolForProxy:@protocol(BackupServerProtocol)];

    [backupServer backup];
    NSLog(@"Work finished.");

    [pool drain];
    return 0;
}

Now run both programs (server first). You can see that the client, connected to the server, should wait for the backup service before leaving the connection.

COMMENTS

Now, imagine that the backup process lasts five hours; it is irrevelant that the client remains connected all this time, synchronously. What happens if we change the backup method to become an asynchronous method ? It is simple : the client will not wait the end of the process to leave the connection. In your BackupServerProtocol, change the backup method as follow :

- (oneway void)backup;

If you run the server and the client again, you can note that the client doesn’t wait for the backup service before leaving the connection. The server continue the work while the client can execute other requests. In this case, we notice that only the interface code is modified, and not the implementation. This quality is directly related to object oriented programming which plainly separate interface from implementation.

Conclusion

Pro’s:

Distributed object programming is a great alternative to basic Socket coding, especially if your are an object programmer. The hard break-up between the protocol definition (interface) and the functional code which implements it (implementation), is a smart way to fit network programming features into the object oriented paradigm. It allows :

  • Fast and clean protocol definition
  • Easy to use and reuse
  • Really takes part of object oriented programming

Con’s:

However, distributed objects are not cool as it sounds. In fact, one of the beautiful thing of Sockets and of their low-level, is their possible inter-operability between different kind of computers and operating systems. Instead, many distributed object implementations are provided by many institutions, which creates a lack of interoperability between systems where Sockets were created to connect them together. Implementations like CORBA provides a large interoperability between several systems, but with the disadvantage of a time-consuming third-party API.

Cocoa Distributed Object API provides a very fast implementation of this concept, but it lacks of a real fault system. If you experiment a little more distributed objects with Objective-C, you will meet a lot of issues regarding remote objects drop : there is no way to know if an object is always bound or not, available or not. A bad thing that relegates distributed objects to a funny API without any serious possible application.

REFERENCES