Update - Great progress

After nights of bug finding, I finally solved the bugs in CoreQuery and it now works. I have also finished coding CoreData. This means I now have the object storage components working and I can allocate and manipulate data objects. Next components to code: CoreMeta - Deals with meta-data of data CoreType - Deals with types (like classes) and syncronization with instances CoreFolder - Provides a generic folder object type CoreFile - Provides a generic file object type CoreProperty - Provides a way to handle properties of an object (meta data) I should finish all these by the end of the week... wish me luck! So far, I am on schedule for the demo. If I see myself late on schedule, I have devised which components will get cut for the demo and which wont. So the original plan is to get all the original components to be in the demo, but if time is short, the demo will use quick implementations to get the demo out of the door a bit like the first Screens version. The demo depending on progress will hopefully look like the UI I have planned for Screens. You will see a few new ideas that have not been implemented in an OS before like my icon slider and multi-mode button. Hopefully I can get those visual effects working as well for the demo but that is pushing it. The demo will use a 320x320 resolution with 16bit color. It wont support HiRes+ because I think the energy should be spent on the insides of the demo and not its resolution capabilities (but it will support landscape/portrait). We will see how time works for me.

Update - Changes

Hi Everyone, What a week... I was stuck untill yesterday on a last function which is to remove data objects but I have solved the problem and hope to finish coding it by monday. Screens is moving along however I have decided to push the multi-threading and module support to after the demo. The demo will be a UI based on the data object storage which will be a launcher. It will launch PalmOS applications utilizing the object storage, so the demo will be basicly a small launcher. The idea is to show you that Screens realy exists and will allow me to be able to test the multi-threading and module support more easily. The demo should be finished before the end of july. I start working on the type system this week and should finish the basic types by the end of next week. Note that the API wont be present in the demo so you wont able to write applications yet for it but the demo should give you a general idea but not how Screens will finally look. The demo will use 16bit color on a 320x320 resolution and will contain an initial design of Screens.

CoreData nearly finished

I have already coded half of CoreData and should finish by tuesday. Things are going well... After CoreData I have the multi-threading components. I already have code which works on how to do the thread switching but I dont know the speed but it should be fast, not as fast as normal thread switching but should be fast enough. I will do tests on the components as soon as I finish coding them. Note that the kernel is only for developers, I will work soon after the release of the kernel on the UI module which deals with the graphic layers, input and window manager.

Why should I use the kernel API?

So the kernel will be released soon but it wont have the user interface APIs, so why should you as a developer even consider the kernel API's? The kernel provides two key elements: The object storage and the thread functionality which are basicly interconnected. As a developer one of the key things we do is store data. This could be a document, temporary data such as proccessing or any data you manipulate with your application. With PalmOS you currently have four options: Dynamic Memory, Feature Memory, Records and Resources. All these API's are very low level and wont do much more than give you chunks of memory and allow you to read/write to them and resize them. Resources do allow you to tag each chunk with resource and typeIDs but nothing more. Screens Object storage provides you with the ability to store data in its structured form without having to parse it in into structures to manipulate it. Each object can be based on multiple types which are like classes if you are used to C++/# or Java programming. You get basicly most of the advantages of object oriented languages into C but Objects are persistent untill you remove them so you dont need to revert to byte streams when you want persistent storage. The multi-threading APIs allow you to split your code into multiple functions that run simutaniously. This is not pre-emptive multi-threading, its cooperative however you dont need to specify yield points because any Screens API call can yield, so its easier to code than cooperative but is not as robust as with pre-emptive multi-threading. Data comes in many forms and I feel that the object storage will satisfy your needs if you check it out. Because you can do the Screens API calls in your current PalmOS applications, its easy to improve your existing application instead of rewriting it. Remember that the kernel is free to use in your products without licencing and that there will be alot of documentation available and email support to help you with integration.

Update

Screens is realy moving forward... Here are the list of changes: 1 . CoreList has been canceled since it wasn't going to serve any other components in any way. 2. CoreObject has been renamed to CoreData because it basicly provides only the data storage in which the methods are built on top of. The public developer API will use CoreObject instead of CoreData (CoreData is a private kernel component). I should have CoreData finished by tuesday. 3. The kernel release data is 8th september 2005. The kernel includes the object storage and the multi-threading API's. This is to start programmers in the programming style and getting programmers to start coding modules like the bluetooth and network stack while I focus on the graphic layering and window manager modules. 4. Betas will be released before hand as soon as I can finish the type components. They will be tested and will be released for looking at and evaluating the API so I can modify it based on developers requests. 5. Once the kernel is released, any developer can start using the API's in his existing PalmOS application. He just needs to include the kernel prc on the device and call the API wrappers (include the header and source files) and thats it. No linking, no libraries. Just include the files, call the wrappers and include the kernel prc file on the target device. The kernel is open-source but the betas will be closed untill the final release (september). 6. The kernel is only usefull for developers. The shell experience will be released hopefully by the end of the year but wont be free. Only the kernel and other API modules are open source and free, the launcher and shell will be commercial.

Finished coding CoreItem

Yes... its happening... I finished coding CoreItem apart from CoreItemAlign but since aligning is not needed at the moment (only at the end of the kernel) I wont code it now. But its not a deciding factor and not complex so dont worry. Next week: CoreList & CoreObject CoreList is a very small component that uses an Item created by CoreItem and allows to add or remove structures or data of the same size and retrieve them. Its how CoreObject maintains its list of children for example. CoreObject is one of the largest components but most of it is very easy to code since its based on the functionality that CoreMemory and CoreItem gives me. We are getting there... So Far... So good!

Nearly finished coding CoreItem

I have three functions left: CoreItemNew CoreItemResize CoreItemAlign They are fairly simple with the new design but they have multiple steps so they are longer than the other functions. But dont worry... its not something I cannot handle. I should have CoreItem finished coding by Monday. Then I will be coding CoreObject. You will now see how the coding goes full speed ahead.

PalmOS will survive

First of all, I would like to thank Jeff Kirvin for his latest 1src podcast: http://www.1src.com/headlines/show/001089.html

I think that Jeff is doing a great job and getting people to know about what the current state programmers are going through. They are having a hard time and someone needs to do something about it.

My post over at SoloMedia about PalmOne's ignorance of programmers and users and my solution for it (which as usual has something to do with Screens): http://www.solomedia.org/forums/viewtopic.php?p=2130#2130

Go and vote which components you find hard to code for at my 1src thread: http://www.1src.com/forums/showthread.php?t=89575

I think Screens is finally getting its purpose more clearly told. I am sad that this is the case for developers but it will be even a better reason for them to move to the Screens API's once I release it since its designed to solve the exact problems they are currently having with PalmSource and because its free I dont see the problem of programmers jumping in. I am working very hard... I have pinpointed the design and coding as we speak the basic object storage. My wife will be off her university in exactly another month in which she will help me with the project (I have been waiting for this moment). She is very talented (but she doesnt believe that) and I realy think she will able to boost Screens forward. So... the hour of Screens is at hand and the question is will I be able to take advantage of the moment... Hopefully... Wanna help me? Chat with me via my MSN Messenger group or via any thread you want... just send me the link (Via my MSN account, my email at zhamilton1@yahoo.co.uk or give a PM at 1src, my user name is zhamilton1, or just post it here.) I wont to get involved in any way I can to understand what developers want and need. Want the API to be easy in a certain area, tell me which area... give me your coding needs... let me find a solution for you... anything... In the comming days I will try and talk about Screens in any place I can find... if you have any links to thread locations to talk about Screens... tell me please I realy want this project to work and I think now is the criticial time I have been waiting for... the right time to let Screens out to the world.

Update - Going forward

Hi Everyone, The 1src chat gave me alot of insight and I have decided and designed changes in how the kernel will be coded faster. First of all, to my understanding, developers are realy having a hard time coding for garnet and therefore I feel that its more important providing a platform for developers than users at the moment. However dont worry, users are also high priority, its just that I think developers need a helping hand. To get the kernel out the door, I have decided to stop redesiging constantly to make Screens better and focus more about getting Screens out the door. I always wanted to create the best system I could and I think I am at the stage where cutting corners does not actually mean a less product. The changes are simple: The API stays a great design while the first implementation will be done as a quick and dirty solution which will in the future be changed. This way I can get developers coding via the API while I optimize the internals AFTER Screens is released. Techinical changes: (Screens is coded with C) I have already finished the quick and dirty implementation for CoreMemory, CoreItem and CoreObject. CoreItem basicly allocates handles in a single array at the beggining of the store and pushes allocations to the end when it resizes the array forward. This is easy to implement and still provides a single fixed handle to any allocation. Each handle is 32bit where 24bit is the index (which means that it allows up to 16 million allocations) and the remaining 8bit is used for a handle count to know if the handle is used by a different allocation. Here is the structure for a CoreItem header: typedef struct CoreItemHeader { Int32 Size; UInt32 Handle; } CoreItemHeader; If the size is less than 0 then the allocation is free while more than 0 then the allocation is used. This allows for allocations to be one after each other instead of using prev/next pointers. To simplify the design, when resizing allocations, its moves them to the end of the allocation stack instead of trying to resize it locally. While this does take a bit longer, its easier to implement and has no changes on the API. The handle is used to update its new address if it is moved. To regain free space, aligning is done where it aligns all allocations left overwriting any free allocations leaving the free space always at the end. The aliging is done automaticly as needed. An object is also just the following header in an item allocation: typedef struct CoreObjectHeader { CoreID Name; CoreObject Owner; CoreList Members; CoreItem Data; UInt32 Flags; } CoreObjectHeader; The name is a CoreID which is a 128bit identifier for unique object labeling. Each object has an owner and members. Members is a list of object handles and can include any objects. If an object is not owned then it is a link. The data handle allows for multiple objects to point to the same data but can use it in different modes like copy & write where writing to the shared data makes a copy and writes to the copy instead. Here are the list of flags: 0x01 - Copy on write (explained before) 0x02 - Owner of data (defines that the object deletes the data when the object is deleted) 0x04 - Hidden (the object is not listed when enumerating usefull for structuring) 0x08 - System (defines that the object is needed for the system to run and should not be removed) Thats the object design... simple but usefull. Its not this layer in which developers interact with but rather with a foundation of components which provide dynamic methods on object, properties, syncronization and type inheritence. So thats a summary of the new design which I am going to implement. My priority now is to get Screens out as fast as possible not to add more features or make Screens more efficient.

User benefits

I have many times talked about the advantages of Screens for developers such as a consistent data model and easy multi-threading design but I haven't talked much about what Screens means for users. Well this is a chain reaction in that if its easier for developers to code applications, there is a better chance for them to code better applications. But Screens is about more than that: Users are becoming more and more dependent on thier devices and as such require reliability. Currently if you forget to save your document even on a PalmOS device, its changes are lost. In Screens this cannot happen because even if you are editing a remote document, the changes are still stored locally. You do not have to fear that if an application dies, because any data it holds still stays consistent. Mobile users want to get to things fast and not have to wait for applications to load thier resources. Screens allows users to have as many applications as they want open and switch between them quickly and easily. This is done by automaticly switching applications resources into memory as needed. Even if a soft-reset or you close Screens to run another PalmOS app and return to Screens, it stays exactly the same way you left it... windows, controls and all. Users want Security, Screens will allow to encrypt objects by the user, so unless you log-in via your user profile, the files will stay encrypted. Decryption is done on the fly so there is no need to decrypt all objects, rather only the object you use. Because Screens is multi-tasking, it can decrypt your objects while still allowing you to work. Users normally work on a single task but sometimes they want to branch out to multiple tasks. Current systems are either single tasked like PalmOS, Windows Mobile and Symbian or multi-tasked like Windows, MacOS and Linux. The problem with single tasked OS's is that you are limited to only a single task at a time while the problem with multi-tasking OS's is that you are forced to be multi-tasked and even if you are only using a single task, it does not mean it will use the entire display surface. Screens solves this by its default being a single task design so you only have one document window displayed on-screen but allows you to drag its title to detach it from the single document model and allow you to move it around. This means that normally you deal with a single task but if you want to put two documents side by side, just drag thier titles to each side of the display and they will align themselves automaticly. These features are not used by any other operating system but provide the ability to 'have the power when YOU want it'. Screens goes further... When you are dealing with a document, you can delete the document even if it is open. Screens is smart enough to close the document window before deleting the document file. You can also rename, move or copy a document even when it is open. You dont need to close it and activate the file browser just to rename the document. With single and even multi-tasked systems you still have modes where you are limited to a certain functionality. Screens has a free modeless design where even when the Open dialog is displayed, you can still edit the document that is currently open. If you close the Open dialog and re-open it, it is shown exactly the same contents, so closing dialogs is usefull not just discarding. If you see the Delete warning, you can always edit your document, duplicate it and then click yes to delete the original. This allows you to start tasks when you remember them even if you only want to carry them out later on. Screens doesnt use scrollbars... it rather uses 'user context in the center' design where the center position is the active position and everything around it is scrolling. So for example: a list control that displays 9 items, the 5th item is the active item. Tap any item apart from the 5th item and it will scroll that item to be the 5th item. This makes scrolling more dependent instead of just a static bar which you drag up and down. Some cellphones use this design in thier menu systems, I am just using it interface wide. While this does mean extra taps sometimes, its more work flow oriented - taps is not what matters, its the work flow that matters. These are just a few of the ideas that Screens environment will give users to finally give them power to take advantage of thier device. The zen of computers is not dumbing down interfaces but rather making powerful features simple. Its always easy to remove elements, the question is can you make elements more efficent.

my 1src chat summary

This morning was an exellent chat... I learnt alot from the users... basicly everyone wants Screens out already (so do I) and most think that PalmOS and PalmSource just are not exiting anymore. So... what I am going to do about it? Well... I am going to put a bit more effort in finishing this kernel since thats the hard part of this entire project. So what can users help me with? Chat, Chat, Chat with me through my MSN user, through comments on the posts here... any questions technical or practical are helpfull. So what can developers help me with? There are many features that I wont be coding for Screens but someone else can code like the bluetooth driver, the network driver, the IR driver, the alpha blending driver and so on. When I finish the kernel and create my web-site I will post the API headers which all any developer needs to do is code the internals of the modules. He doesnt even need to send me the code, just the prc module. There will be many device specific features like the Sony Hi-Res which I wont be coding for either but any developer can create a module for. Thats the whole idea of Screens, its a community of developers and users which make small contributions via modules without exposing thier work. So let me just refresh to you that Screens will work on a Palm III and anything above it. The device specific features are implemented via add-on modules so for example sound is implemented through a sound driver for OS5 devices. I realy feel that the kernel will be out by the end of the year. I am further in code than I was ever before, so I think of that a sign that I am into the good object storage now. I should finish CoreItem any moment now and then go on to CoreObject. Note that all these components are already tested, so there are no suprises there.

The object user experience

I get alot of questions of how will the object storage affect the user experience. If anything is an object will users have to sort the properties of a file, will they even see the properties of a file as an object? Will documents be browseable? Will appointments have folders that hold the address fields? and so on... Hopefully this post will explain and answer these questions: First of all... the user and developer experience are different. Users use the UI while developers use the API. The user experience shows folders and files much like any file system does. It just adds many more types of 'files' a user will see like appointment and contact files but they still feel like a file that holds data. Only that it holds structure and data, not just a stream of bytes. Think of a file like a zip file... it holds folders and files inside it only each file can also hold other folders and files in it. Users see a contact file... they can delete it, sort it, filter it and do anything you would expect to do with a file however the developer using the API can dig deep into the contact file to view its properties, members, addresses, relationships with appointments and so on. So only objects which are self-holding and that users use like documents, email, PIM objects, devices, drives, folders and normal files are viewable by the user. All the rest are handled by developers and are visible either through specific UI's like the Properties view or used internally for the developers use. So for a user, there are folders which organize objects. For the developer anything is an object which can hold data (or share it) and hold other objects. So to the user it feels like a normal file system while to a developer its a data storage. Even applications appear to the user as a single object while to the developer - settings are stored inside of it including code and resources the application needs. If a user copies an object like a document... it copies anything in the document including embedded documents. So the user who is used to a normal file system continues to get the same feeling. Relationships with other objects are not copied, but still are referenced. If you move a document to another system, all its files are valid but its external references are invalid and removed unless you make them re-searchable by providing keywords or some other way to find thier new relationships on the new system. So you could make a file relate to a contact by its name so that when you move the file to another system, it relinks the contact by finding it by its name. This means moving objects between systems still remains intact. When you export an object, it includes all the contents of the object in a single file so you can move it between systems even over systems that dont understand the object format. But what about Search? Well... because it is an object storage after all... finding objects of the same type are instant since they are referenced by thier class (so the class can update its instances dynamically). Filtering, Sorting are all user interface issues, not object storage issues. And if you are searching lets say by Name, it returns not the Name object itself but a parent 'user-viewable' object in the results. So searching for a contact name, does not return the name property but the contact file object. Just like you can share object data between objects, so can you share objects between objects. This allows one object to appear in multiple places where only if the object exists in no locations, is the object removed from storage. But from the developers point of view, each object has only a single owner so while objects can appear in multiple locations, its still only owned by a single object. What does this ownership do? Its a spine for the object which defines its true location. When a user copies an object, it duplicates the contents of that object to the target location. If a user shares an object, only its link is copied, not the contents of that object. If the user deletes an object, it only deletes the link. If an object has no owners then it is removed from the system along with all its children (unless they are shared). This removes the need for shortcuts/shadows/aliases/soft-links but still gives the functionality of allowing to access an object from multiple locations. Note that you can always find all the locations an object is shared by to remove it from all those locations. Hopefully this should answer some of your questions about the object storage and its user experience.

Update - Started coding CoreItem

OK, I have started to code CoreItem and so far this is the easiest and simplest implementation I have coded so far for CoreItem. Hopefully I will finish it by the end of next week. I have increased the overhead of memory allocations from 12 to 14 bytes so that the size of an allocation can be up to 2GB. Note that this size is only in the implementation of storing objects internally and that external streams can be up to 4GB (32bit address space). The advantage of CoreItem is that it provides a fixed handle for any allocation which does not change even if the allocation moves around the storage stream. While I am coding this code for a Windows Console, only CoreStream needs to be recoded for the entire base object storage to work on PalmOS or any other OS. Although all meta-data and documents will be stored in the object storage itself, stream based objects like multimedia will be external files encapsulated as objects. So to the developer or user there is no difference but internally there is a difference which is the whole advantage of using dynamic methods on objects to overload default functionality.

Update - CoreMem coded

OK, So I finished coding CoreMem apart from the functions CoreMemResize and CoreMemAlign which are dependent on the CoreItem functions which I will be coding next week. This are going well at the moment... So far... So good... No problems in the object storage design Hopefully (as usual) this will be the last time I need to code these components.

Update - The memory component design is ready to be coded

So, I have finished designing the memory component, so I plan to code it in the next few days. Its faster, simpler and much much smaller overhead per allocation - 12 bytes! Why does it make a difference? Useless on a few allocations but when an average system will have no less than a few thousand allocations, those little overheads can make a difference. Just to give you an idea of the optimization I made in the design: it was around 80 bytes overhead before hand. But optimization was not the reason the memory component was re-coded but rather to make it more easier managed and easier to align allocations. Once I finish coding it, I will post an editorial on how it works which is quite simple but cool. But as usual, there might be something I missed, so I might have to get back to the drawing board. Wish me luck!

Update - Code untill I hit the wall

So... I am trying to code the kernel again however this time I am coding it as a windows console app first. Why? Because its more light-wait to test, the code is cross platform (so I make a few changes and it works on PalmOS or PocketPC) and I can print debug output or comments much easily. It also takes less memory on my limited 256 MB PC. The platform specific content is in a #define per platform so just define PLATFORM_WINDOWS for Windows or PLATFORM_PALMOS for PalmOS or PLATFORM_WINMOBILE for Windows Mobile. OK... Here is some more technical information: Applications can not only be launched by the kernel threads but can also be made to be launched by PalmOS directly. For example, the Screens Environment distribution is a normal module which can be launched by a normal PalmOS Launcher and it then calls the kernel. So the kernel is mostly the first sub-launch. This is how Screens will evolve at the beggining... PalmOS Applications will call the Screens API which will call the kernel. This can allow any PalmOS application to use the advantages of Screens without having to evolve to the programming model of ScreensOS. The idea is that Screens native modules can be launched by any Screens native or not-native module/application however non-native applications cannot be called by the kernel if they are not the active PalmOS application because sub-launches lack globals. You can solve this by allocating your own global space but then you have to save the globals to the thread and restore them each launch which can realy be slow. Screens kernel will be released open-source for multiple platforms (starting with PalmOS) but then I will create extra modules such as the type system and the user interface which will also be open-source. So how will I earn a bit of revenue at the beggining? I will create an MS-Paint like paint application which uses the kernel and other modules as its back-bone. The app will probably cost around 15$ and will feature all the MSPaint features like a thumbnail movable window, nearly unlimited zoom, easy color pallete usage and a pluggable design for adding effects, formats and brushes. This will give programmers and users the advantage of using Screens as an API. The advantage is when Screens will be finally released in its full environment, all these applications that use the Screens API (and either dont use globals or allocate them for them selves) will be able to run inside the Screens environment directly.