Update

As you can see I am writing alot of information in editorials lately just to give you another slice of what Screens has in the kernel core. I figured out how modules will interact with the kernel using FtrGet and SysAppLaunch which are the only two PalmOS functions used by user modules. system modules can use any PalmOS functions to implement functionality for other user modules to use. The idea is that system modules are device dependent while user modules are device independent. Think of system modules like drivers. There are still a few issues on how to actually store objects in a database which I hope to solve soon (as usual). The issues are like to do I use the database like a file stream where each record holds data of a fixed size or does each record represent an item. Hopefully I will be able to talk with some of the MSN friends in my Screens group to get some answers. The object storage is evolving very nicely and so far I am very proud of it.

Synchronization (Editorial)

Synchronization is a great feature in the object storage. Its how types work, how extensions work and many more features. First lets see what it could be used for: 1. You will be able to use synchronization to view a workspace on your device display and on your desktop easily even though the desktop has a different resolution. This is like VNC however Screens sends the actual drawing primitives instead of bitmaps, so its much easier to deal with. You will also be able to do separate log-ins to your device. Here is the vision: You are writing your document on your handheld and you go to a internet cafe, via the web, you log into your device via Wifi and continue writing your document on the nice 1024x768 flat screen monitor. Then you just log-out and continue writing the document on your handheld. It allows turning internet cafes into terminals. Think how cool that would be! 2. You will be able to track changes and work with a friend via Bluetooth on the same document. You make a change, and he sees the change. He makes a change, you see the change. You could also make team changes and decide which team members can modify the document, which can only read the document or which can make changes only to a copy so if they make a change, its not making a change on the original document, but rather on a copy. 3. You will be able to specify web-pages which synchronize locally so you can view them on your device even when offline. But you can take it a step further and specify to synchronize only when the connection is not in use, so it does not slow down your normal browsing. 4. You will able to synchronize playlists or collections of files to your device depending on appointments. So you can make an appointment repeated monthly which shuffles and downloads 80MB of your music on to your device, swapping the existing files apart from your favorite songs which stay on the device until you decide to let them swap as well. You will be able to specify that a presentation should be on your device a day before a business meeting and that it can be removed a day after the meeting. These are just ideas of how flexible syncronization could be used for. How does it work? Simple... every object which its data changes or its structure changes, an update message bubbles up to the root object allowing any parent object on the way to handle the method. This is how types work - Since all types are under the 'Types' object, any changes to any type bubbles up the update message which the 'Types' object handles and syncronizes the change with the type instances. If data is shared by multiple objects, they all get an update message rising up to the root object letting any parent handle the change via a method.

Why is Screens needed anyway?

Many times I get this question... why will Screens help me? Well... I now found out the hard way why Screens is needed for developers: API The PalmOS API is very low level, you deal with pointers, allocation, database locking, bit manipulations and everything is call this first then this or else... booom I am currently writing a normal PalmOS app for someone and when I found myself searching for how to handle the resolution change not even control moving and modifying... just knowing when changing into portrait/landscape mode and when the DIA is open or closed took me a long time raging over google, PalmSource and PalmOne for how to do it. These are simple things which should exist in ANY application... I am not talking about the Serial manager or bluetooth, I am talking about a frickin display basic manipulation. Sony... they realy stabbed everyone for buying thier devices by shutting down thier developer web site so no one can make apps for thier devices anymore. Smart move, Sony! And to try and figure out how to handle the T3 slider open/close, I still dont know how! Screens is going to have a very generic API without trying to be device specific. Its more like windows where you have a a generic model and have drivers to do the actual implementation. In Screens, you will write your application once, and it will automaticly use the driver its meant to use on each device. So a T3 might have one driver while the Sony device might have another driver. But the API and the applications stay the same. You think that the whole idea of an OS is to bridge between hardware, PalmOS sucks at doing that. There is a need for someone to take this forward and I dont expect PalmSource to do it, because it wont be compatible with old PalmOS versions! Which is why someone like me or maybe someone else will make a convenient, simple to use, flexible and consistent and device independent API which developers can use once on the simulator without worrying about testing on all devices.

Semi Pre-emptive Multithreading

Screens has a semi pre-emptive multi-threading scheduler. What is the difference? Single threading: PalmOS is actually pre-emptive but you can only have one application which has only one thread running. You cannot create threads however you can use the existing threads like the multimedia thread and the serial manager thread. Cooperative multi-threading: Windows 3.1 and MacOS up to version 9 are cooperative which means that you can have threads but they each thread must have points where it gives up (yields) control to the scheduler so that the next thread can be continued. This means that threads must call a specific yeild function at many intervals or otherwise it can starve the other threads in the system for CPU time. pre-emptive multi-threading: Windows 9x/XP/2003 and MacOS X are pre-emptive which means that you can have threads but the system forcefully yields threads instead of threads deciding when they want to share CPU time with other threads. semi pre-emptive multi-threading: Screens is semi pre-emptive which while in reality its cooperative in that the thread does the yielding, any API call can yield so as long as threads call API calls, the system can switch between threads. This gives the advantages of pre-emptive in that you dont need to structure your thread around yielding points but a simple for loop can still freeze the system. Why use the last method? Screens runs on top of PalmOS and it does not have access to the multi-threading API (only the system apps can). There are a few advantages of using semi pre-emptive multi-threading: 1. The threading actually happens in the module, so the threading is language and implementation independent. 2. You dont need to implement locks, semaphores or mutexes or deal with interupt problems. Screens does not use these. Because modules dont have dynamic memory, the sharable memory is the object storage and the object storage does the blocking of threads for you so that no two threads will read/write to the same object. So what is actually happening? The kernel (Screens Core) loops between threads depending on thier priority. When the kernel resumes a thread, it just calls the threads code application (prc file). The application then restores its stack from the object storage using DmFindDatabase and SysAppLaunch to do kernel API calls and jumps to the stack using ErrSetJump and ErrLongJump. When you do a kernel API call, it returns non 0 if the thread needs to save its stack and close down. Note that example code is given so developers dont need to do this, wrappers do it for them. Kernel calls are done using SysAppLaunch sub launching however most of the time function wrappers you call are doing a combination of kernel calls and might be doing a context switch. Developers do not have to worry about the swithing mecanism, just do API calls as normal, the example code does the rest. While calling a PalmOS function does block the kernel untill the function ends, the Screens API calls can yield to the kernel at any time.

The object storage - continued

The object storage has a few depths: 1. The data core - This provides the actual interface and implementation between the storage of the bytes (the PalmOS database or file stream) and the actual object design. It supports the ideas of a tree structure of objects and links, objects that point to data and data which points to multiple items. This indirection does mean that its slower but the flexability is just huge and allows to change the implementation in the future. 2. The dynamic core - This provides the thread and method functionality which adds the ability to connect code to data through methods. Each object can have methods which define functionality for it. From predefined methods like Open, Close, Read, Write, Move, Copy, Link and so on to custom methods like the window class has the Draw method. The programmers API does method calls, not direct function calls to the underlying data core. 3. Types - data has a common style which is known as the data type like windows, controls, layers, appointments, contacts, strings, numbers, bitmaps, resources, users and so on. Each type is an object just like any other object but it has methods which allow to syncronize between the type and its instances. So adding a new property to a type, adds the property to all instances. These methods are not defined per type but rather in the generic 'type' object. All types exist under the 'type' object so they automaticly get syncronization. So adding a opacity property to the window type means that all windows will have the opacity property and they also get the default value specified by the window type in which each window can override. The object storage is very simple and flexible which means that it can be used for nearly anything. Screens will be importing/exporting objects as XML so you will be able to export your document from Screens and move it to any OS and read it. I see the power in this and I am very happy with it. No, its not implemented yet and its taken me 3 years to get to this but the advantages I see with this design are just so worth it. As time goes by, you will see why.

The object storage (Editorial)

OK, This project is taking alot of time on just the object storage. Why not just make a plain file system (folder+file) and get Screens out the door? This is a constant question I have got and the problem is that people dont understand why this single component is the most important in this project. Data - Its all about Data. Computers manipulate data thats all. You have different storages like RAM, Hard drive, Disk-On keys, SD Cards which hold data. The computer or device manipulates that data either by reading or writing to it. A computer is useless without memory because you couldnt do anything with it. So data is important but whats wrong with a file system? Lets go through the changes that Screens object storage improves over a normal file system: 1. A folder can be a file and visa versa In current file systems, you have folders and files where folders provide structure and files provide data. In Screens, an object is both a folder and file, so it can hold both structure and data. How does this help? Well... its like bundling. You can move an application (data) to another device and it includes all the structure and data of the application with it. It gives more flexability and makes information more accessible. 2. Links are direct manipulation Unlike shortcuts which are basicly files that direct you to the real file/folder, links act exactly like the object it points to apart from moving, removing or copying the link which just copies the link, not the object it points to. But anything else affects the link directly. This means that you can use a link anywhere you would put an object the difference is that links point to the object, its not the object itself. Using links you can make the tree structure of objects more flexible. 3. Links are like aliases and hard-links Links point to objects by thier handles so it does not matter where the object is moved to, the link stays pointing to the correct location always untill the target object is removed in which the link is removed as well. 4. Data of objects can be shared over multiple objects with different access policies. Instead of an object exclusively owning the data, it can be shared over multiple objects so each object feels that it owns the data and can manipulate it but the data is only removed when no objects own the data anymore. However each object can have an access policy where it can be allowed to only read the data or have read and write abilities or it can be allowed to only write to a local copy which is called Copy & Write. Copy & Write means that you can read the shared data but if you write to it, it creates a copy of the data and writes to the copy. If the object is the only one who owns the data, it does not make a copy. 5. Object locking Threads can read data freely but only one thread can write to an objects data. All other threads are paused untill the exclusive hold of the thread on the objects data is released (which is when the thread closes the object data). A thread also has to wait untill all threads give up read access before it can get exclusive write access. This makes sure that no two threads will ever read and write at the same time which is a common problem in multi-threading environments. This means that developers dont need to deal with semaphores or mutexes, the system deals with this for the developer. 6. Objects have operations Each object can have operations called Methods. Method are ways to manipulate data using module code. For example an email has a method called Send which sends the email. A music object has a method Play to play the music in a multimedia application. 7. Objects have types Unlike extensions, Objects can be based on multiple types so an XML file can be based on the XML type which is based on the text type so any text application can open the XML file and also any XML application can open the XML objects. Types also allow to provide default properties for objects so for example the window title color can be made a default value in the window type and any window which does not provide a different title color will get the default window color. This is just a tip of the iceberg of what Screens object storage will able to do. It also deals with threads as batch jobs where only in specific locations do threads actually write the object storage which allows for application crashes not to leave objects in inconsistent states. It deals with versioning, extending types and object properties, members and data. Its very simple design allows to be flexible and powerful and still work on handheld devices like PalmOS and PocketPC devices.

Update - I think I have solved the object storage again

Yep... Again I have a design that I will go over the next few days to see if there are any problems with it. So far its a very nice design which is sort of a merge between a relational object storage and a file system structuring. I'll write an editorial on how great it is and why it is so important when I find that it suits Screens.

Update - Problems with the object storage

I am having problems with the object storage again but once I solve them I'll try and code the CoreLayer component again.

Update - CoreLayer

OK, So I am coding today CoreLayer. CoreLayer is quite a big component compared to the other components I have been coding in the last two weeks. It deals with the actual layer objects, updating and drawing the layers to the display. CoreLayer is quite a good design since it allows transparency with ease with only one offscreen buffer. Layers are like markup and store the points, actions and commands that layers use to draw like 'draw rectangle at position 0,0 and finish at position 0,0'. When a layer wants to be updated to the display it calls CoreLayerUpdate which adds the layer to the update layer region. It basicly enlarges the region to be big enough to hold all update locations. So if I only update layers in a certain area of the display, only that area is refreshed. CoreLayerDraw does the actual drawing of layers clipped to the update region. When it draws layers, it draws them one on top of each other on the offscreen buffer. It of course uses clipping to increase performance to not draw uneeded parts of the layer. It also only draws from the first base layer and on top. The base layer is found as the first layer on-screen which covers the entire update area and that is not transparent. This basicly will allow me to easily get transparency effects without having to use a mask. It acts more like a pile of papers where it draws the lowest paper and draws each paper on top untill it draws the top paper. While the CoreLayer will only draw basic primatives like lines, rectangles, round rectangles, frames, pixels and other PalmOS drawing primatives, it will expand to support much more complex types in the future.

API Perfection

You constantly see me making changes in the API design even before I have actually coded them, or even recode an entire component just because of a small change. You might think that I making changes just for the sake of it but there is something much more deep in this. If you look at many other development API's they have a constant problem of not being usable easily. Some use low case letters only which means that you have a real hard time reading your code when it uses function names like gtk_signal_connect or gtk_box_pack_start. C++ API's also have problems that thier class names and methods feel like programmers just added them in one by one without caring for any consistency. Just a look at MSDN .Net classes and you instantly get puzzled. I know that the Screens API is going to have problems of its own (no static type checking since object handles are actually UInt32 typedefs) but I did want the component names and function names to be very consistent and easy to use. I am constantly talking with my Messenger users about function names like should it be CoreListCount or CoreListSize to return the number of items in a list. They sound similar and you would think that Size might be in bytes but then again its a list and when you say the size of the list you mean the number of items, not the size in bytes of the list. Another reason I chose CoreListSize is because it has a 'set' function called CoreListResize which CoreListRecount doesn't work well. Nearly all function names have consistent names like Read/Write, Open/Close, New/Free, Get/Set and you will able to find them across most components. I myself plan to code alot using my API (My API uses itself, a brick on brick design) and I want it to be easy to use and not constantly grabbing some online documentation or book to write some simple code. I want to be able to see the function definition and know how to use it without special notes and warnings like you may only call this after this or that after that. There are times when these are crucial like you need an Open before you do a Read and you should not use a Write after a Close but even then your code wont fail but rather will reopen the stream for read & write use even if you only planned to read from the file which means that any other thread which wants access to that stream will have to wait untill either the thread is released or a Close function releases the stream. But its easier to add in an Open/Close function pair then finding why the application dies when it reaches the Read function. Programmers will be able to be notified of this via thier module blog which is where errors and warnings can be filed to the programmer without interupting the user. Over time I will probably continue to modify function names untill I get them right, but its better I spend time renaming them then programmers spending time being frustrated.

Update - The usual changes

OK, So here is the list of changes I have made: CoreRectangle is now CoreRegion because it the name fits its description a structure holding left, top, width, height positions. The entire UI module components will be Core components for a simple reason. It takes a context switch for each external module call (apart from the Core kernel components). So it would take much longer to do simple drawing to the display if it had to switch every time it wanted to draw something on-screen. Also although UI sounds seperate, it realy is not since every system needs some kind of input/output system, Screens is based directly on a GUI (Graphical User Interface) instead of a CLI (Command Line Interface). I dont see any problems coding this week apart from CoreLayer (used to be UILayer) which might be a little hard but I should be fine.