The search for storage Part II (Editorial)

Part II: Default Consistency In part I, I talked about that the object storage needs to be simple if it wants to even be implemented, not even talking about using it. I do alot of research and here is one of the articles called "The Anti-Mac Interface" http://www.acm.org/pubs/cacm/AUG96/antimac.htm which goes over alot of issues but the one I want to focus on is consistency. While complete consistency is quite impossible if the storage is going to be useful (if it was totally consistent, there would be no such thing as links because deleting an object would always remove its data) but thier are basic rules which are needed to be kept especially in the removal area. Having removing an object in one place, removing it from another does not sound like a good idea and many times I have created a shortcut to a file instead of a copy just to find it out when I delete the original. Ways to solve this is by using references where removing the last reference, removes the actual object. The problem with this is that current reference designs are very limited where links are either included in the reference count or not. WinFS is built this way and its not even released yet. The reason this is a problem is just a plain idea of categories. If I put references to a document in multiple categories and I want to delete the document, I dont want to have to remove it manually from all categories. A way to solve this is by allowing objects to list thier references and then just remove all references, but then you have lost the whole idea of references in the first place which is to allow data to exist in multiple locations shared. So you need a way to have references to references where you can have basiclly a tree of reference points so deleting the last reference of the original object will delete all the child references. The problem with this is that its not very consistent. Why should it be? That is exactly what the Anti-Mac article talks about. These known things like "the user should be in control" or "Metaphors" are what cause problems with this design. If I reference to a reference which references to a reference which references data, doesn't this sound a bit complex. Well yes... but I dont normally need this depth of referencing anyway but it would be useful if the time came up. How has this got to do with consistency? Not much... just kidding... The thing to understand here is that sometimes consistency can limit you. I am not saying that things like the OK & Cancel buttons should have different positions in different dialogs, I am saying that consistency should be a default, not a forcing rule. Think of it that consistency is the way a bar of metal should be, straight and strong but bendable if needed to be. This comes to the first word of the title of this editorial which is Defaults. Defaults are a very powerful feature. It allows a normal user to use an application or system and feel that its small and comfortable but if he wants he can unlock the power of the system when he needs that power. Users dont like modifying techincal options very much. They might do some tweaking for performance sake but most of the time the options you want to modify have a look & feel written all over them. These should be the default options displayed consistently throughout anything. Dont tell me I need to enter a port number if you know the one to use in the first place. I might want to use a different one, but since 'I might' is less important than 'I mostly use', the default should be non-technical options which make changes in the way I work, not in the way the computer works. This design should be applied to the file system as well. I dont need to care about providing a 'save' name to every document just because the file system is dependent on it. Names should be optional. If you are dependent on some unique name, create one yourself and allow the user to change it LATER IF he wants to. The same goes with the documents location. Dont force the user to search for a location if you know the most common one to use. No, Its not the last used location because probably I chose a different location because I made an expection, not because I wanted to make it the default next time I save a document. It drives me nuts when I save a new document on a floppy but next time I save another new document, it shows me the floppy to save it in! Floppy is a temporary storage and you dont create multiple documents just to save them to the floppy and if that is the matter, allow the user to jump to the last used location but the default should still be 'My Documents' or such. Defaults can be a great factor if a user is happy or not. Before you prepare to make a user make a decision or choose an option, think about if it is realy and totally neccessery to ask him. Its like these installation wizards which most of the time you press next, next, next, next, next and another 7 times next just to install a product where it should show you a single finish button with a great button called 'Options' on the side. Now wouldn't that cut the need for a wizard just to install applications.

The search for storage Part I (Editorial)

Storage... The hot topic which is nearly always vaporized into the future but never quite gets into the present. But what's so hard? It sound simple doesn't it? Why is it that so many object storages never get delivered (WinFS, Screens Object Storage, Cairo Object Store, Gnome Storage...)? The file system was a very basic design where it did x and y and nothing else. It had folders and files with basic meta-data and although you had NTFS and BeOS with extra attributes, they still were a file system. That is the success of the file system - that its simple. It might not be what people want, but its simple enough to explain and implement. Object storages on the other hand try to be the 'perfect file system' and promise to solve every file system problem you have like file location independent, description of file contents, search, filter, sort, slice and dice your information in any way you want where ever you are. And this search for perfection sadly never goes anywhere because nothing is perfect and designs that sound good not always work out when you need to write the implementation. You suddenly find huge performance bottlenecks, problems with compatibility and scalability over unknown systems and you get frustrated and in the end give up or push the date of release further and further away. No one wants to be the middle step between the file system and the perfect storage because of fear of obsolete and then we are still stuck after 30-40 years with the same basic file tree structure with basic meta-data. But is this really bad? Does the actual file padigram need to change? This editorial checks a bit of this out. Its too long for all of it, so hopefully I will break it down into multiple parts: Part I: Simplicity One of the basic things we first do with data is store it somewhere. So where do we store it? Well... in a file system we store it under a folder. But wait... what if I want to store it under multiple folders? That would be cool now, wouldn't it? The ability for your files to be location independent, allowing you to access your files from multiple locations is like being able to get your money from any bank. Sound wonderful, but then what's the problem? The problem is quite simple. Computers are very fixed, they do X and Y and if X fails, you must know how to handle it and the same with if Y fails, you must know what to do with it. So while storing a file in multiple locations might be handy for a user but if I want to open a document, where do I open it from? If I delete the document, does it delete it from a specific location or from all locations. While a user can decide on this, computers cannot. Computers do linear actions and cannot decide by themselves. You must tell them where to look. So, lets take the bank example... Where is your money REALLY stored? In your specific one local bank you have your account in. The other banks just reference your money and transfer it over to them but your whole account exists only in one location in your single local bank. Lets apply this to the file system... Your documents exist under a single location but you can reference them anywhere. Sound familiar? Shortcuts, Aliases, Hard links, Soft links are all solutions to referencing your data. So where is the problem? Well all these solutions fail on certain areas and transparency is one of them. There should be no real difference between using a reference and the original apart from three actions: copy, move and delete (move is actually copy to new and delete old). Apart from those actions, everything else from open to properties should work on the original. That sounds good, and its implementation is not that hard to implement since you are basically mapping all methods apart from copy, move and delete to be passed on to the original object. But... Lets take this a step further... If I have a document in a specific location, I might want to associate with it location specific information. For example: if I have music file in one playlist, I might want it to be listed in a different position then in another playlist and since the position is stored in meta-data, what to do? Well... you make the reference become a derivation from the original, so the reference shows the original plus any changes by the reference. So the reference becomes the original+reference. So now what's the problem? Well... if I add data to the reference, its not going to modify the original and then what's the point of the reference in the first place. This just shows you how a simple modification can kill the entire idea and that's one of the problems we face when creating the perfect system. We are not perfect people and therefore cannot understand perfect ideas. We like simple solutions, not complex ones and so far, these proposed object storage systems have been more complex and full-featured than simple and basically become a bloated design which grows until you just cant understand what's its advantage in the first place. Tune in for the next part when I can get round to writing it...

The bad and the good

Hi Everyone, This is already my second week in my new job as a programmer in NDS (www.nds.com). I am working on a project there around to the end of september in which my they will consider if I will continue in NDS or not. I am moving to my own apartment (I am currently renting) next year hopefully and I need to save up some cash. Instead of posting 'give me money' links, I decided to get an extra job, so I now work two jobs: NDS and the escort of 'special' children which should bring me a decent salary for a while. So whats going to happen with Screens? Well, yes, it will be postponed a bit, I dont have much choice in the matter since I am working nearly all day (my escort job starts at 6:30 AM!) but I am trying realy hard to keep Screens afloat. But I have talked with some NDS people about Screens and they are going to try to help me pull Screens from a small one man project to a sponsored project. So there might be help after all. I know many of you feel decieved, but I prefer to face you with the facts then just disappear. Life being married is wonderful but its great responsability and had to be my highest priority hence the need to earn more money to get an appartment than working on Screens which brings me zero income (I know that its an investment). If you have noticed, I have hardly even been online on my computer since mostly I have been working on my laptop (that I got from NDS). When you have seen me online, it was probably my wife who is finishing university on the 17th of july. The bottom line is that I am trying realy hard to juggle all my jobs together, so just understand that reality does bite a bit and our plans most of the time dont work out as planned. I could say, well... I have been promising Screens for 3 years and its been totally vaporware untill now, so therefore I should stop... but then I feel it would be a waste of your time. If I do ever release Screens, it will make all your efforts suddenly worth while.