Update - The train is moving

I finished coding the following components which means nothing to most: CoreObject, CoreMember, CoreFolder, CoreProperty, CoreRectangle, CorePoint and CoreNumber Next week I hope to finish: CoreDate, CoreTime, CoreString, UILayer, UILine, UIRectangle, UIPixel and UIColor While this does not mean much to programmers and users let me explain what this means when I finish those components next week. I will be able to put the code in some prc and give it out you all so you can see that I am definently working on something. I'll probably do some kind of animation with the layers so any simple animation ideas are welcome (that use line, rectangle, pixel combinations). The Layering engine uses one single off-screen buffer where it paints the contents of the layer clipped to the buffer and then copies the buffer to the display. Note that it does not refresh the entire display each time, rather only a clipped area which needs to be updated (called the update region). Becauses it paints layers contents one on top of each other you get natural transparency without using transparency colors. Any area not used by a layer is transparent.

Update - Change of direction

Because of a few events that happened this week I will be putting the multi-threading on the side (now that I know it works) and I will be working on the object storage itself. I think its time to actually get this object storage out the door. Which is why I am going to code like furry this week and next week to finally get this bit ouf of my way because while it has an important aspect, its taking too long.

It Works!

OK... The multi-tasking works where it can switch between two threads concurrently. I now will take the code and encapsulate it into the CoreThread component which I will code this week. This means that Screens will have threading capabilities which is great! The kernel will use this threading capabilities to start the module calls so that modules can run concurrently. This is done by a module saving its state (registers and stack) when it yields (which a kernel API can flag the module to yield) and then closing itself. When the kernel re-runs the same module it passes its state back so that the module can restore it. Because the saving and restoring is done by the module, its not interface dependent or even language dependent. I am so happy with how the project is moving in the last few months in both code & design. Note that each thread has its own stack of module calls where each call in the stack has its own state. The current implementation uses ErrSetJump and ErrLongJump to save/restore the registers and copies the stack between an initialized variable at the beggining of the stack and a variable at the end of the stack. It will be currently stored in feature memory. While this is slower than setting/getting the A7 register, its more implementation independent and wont break on cobalt which many other implementations do.

Transactions (Editorial)

Many are experiencing the problems with NVFS on the new Tungsten T5, E2 and Treo 650. The problem lies in how PalmOS handles databases and how it was not ready for such a back store design. When PalmOS was first written, they chose a combination of ROM (Read-Only-Memory) and RAM (Random Access Memory) to provide storage and dynamic memory in one. They used ROM to store the data which you would find in your device after you bought it more like factory settings. RAM on the other hand was split into two parts: Dynamic memory and Storage where dynamic memory was read/write directly while reading storage was just like dynamic memory but writing had to be done with a special DmWrite function. The advantage was plain obvious. Instead of having a file system, PalmOS just has memory chunks chained to each other which form the databases and records we all use. However there was a problem with this design. While it was wonderful in performance terms it was too direct. If I wanted to lock a record, I would get a memory address which I could ready with direct manipulation and if I wanted to write to the record, I just pass the memory address to DmWrite. If the record memory address moved (when unlocked), I would have to get its address again by finding it through its database-record relationship. So where is the problem? First of all... Each internal card slot (which will go away with Cobalt) has a maximum of 256 MB. While this was a far away limit with the first Palm Pilot it sounds alot like Bill Gates when he said that 640K was enough for anyone. Limitations in a design show in the end in one way or another and PalmOS hit that limit. We now have devices with 128MB and they knew that they had to solve this sooner or later. So how could they solve it? Well... along came NVFS which is a Non Volatile File System which is basicly like an internal SD Card. But unlike RAM, you cannot read/write directly, you have to do buffer based reads and writes much like you do with VFS. So how to make current applications compatible? So they basicly made it work like a desktop system (and PocketPC) where you have real RAM which is like your active working memory and a seperate storage which like windows has FAT/NTFS, PalmOS has NVFS. When ever a programmer opens a database, it copies the database and all its records to a temporary location in RAM and writes it back when you close a database. The problem was that many programmers took advantage of the direct read/write and were reading/writing to records even when thier databases were closed. The second problem was that the amount of RAM they used as the temporary memory for databases was only about 10MB which means if an application opened two many big databases the application would burn and crash. The temporary amount of RAM will grow and grow just like the desktop systems do but then you will get the famous saying 'there is never enough RAM'. What is needed is for developers to move out of the direct read/write design and move into a transaction design. A transaction is a group of actions which are executed at once. So instead of reading and writing directly where if the application crashed in between writes you would have an inconsistent database or record state, you would specify all the operations and then do all the write operation in a single push. What are the advantages of transactions? 1. Because writing is done in a single action, you have more centralized points of failure and it allows your application to be either working or not working without the gray area of inconsistent or invalid data. 2. Because reading and writing is not done with direct memory addresses, it allows the amount of store to grow without having to increase the temporary memory because nothing is directly accessed rather you call a function to read some data into a buffer (which is stored in the temporary memory but is much smaller than the entire database or a group of records) and another function to write data back to the store. 3. We will able to take advantages of mini hard drives or remote store like an FTP site. So instead of being limited by MB's, you have Gigabytes of memory to your use. So what are the disadvantages? Speed Buffer based read/write are always slower than direct memory access. However depending how you handle your memory its not that slower. I dont have benchmarks but I believe that speed is less of an issue when it comes to reliability, expandability, flexability and so on. I believe that if PalmOS goes more towards this direction, they will save themselves many problems in the growing storage direction and more reliable need of the system.

Damn... no luck

OK... so its not going well... but I am optimistic... I found some article on cooperative multitasking which is nearly portable so I will be coding/testing it out next week. This is realy important feature and I am working basicly day & night to get this done as soon as possible, I dont want to take another few months just to solve this. So lets put this aside for a minute and talk about the UI: So what do have in plan for Screens? Its nice that I expose all these programming stuff but what realy interests users is when will it come out and what will it have/do. The first thing I have been terrible at predicting so I just stopped doing so (It was meant to be released 3 years ago). However I can shed some light on what I at least plan to be usefull in the UI which I am always refining. First of all, Screens is going to be very search oriented so for example alot of functionality is built in the basic controls. All list, tables and tree controls are searchable, sortable and filterable. For example if you start typing letters while a list, table or tree is active, it filters out the entries in the list/tree/table to only show matching results. For a tree, it expands the items but still exposes thier node structure. So you can know not only the result but how to get to it. This makes it very usefull when dealing with structured information since it can stay structured and you can search through it. For example when you want to open a document, it shows you a browser dialog however instead of having to expand the tree items to find the document you want, just type the name of the document (or part of the name) or any other meta-data which matches the document and the tree will filter out any non-matching entries so you can find your document much faster. This is going to be an addictive feature because its not obstructive and still keeps structure instead of hiding it like other methods (queries). I'll post some more of my UI ideas soon

Update

OK... So I have trying to debug the multi-tasking code yesterday and today and no luck. I just cannot call MemMove from and to the stack while its in use. I will use a different technique which is the A7 register stack pointer. Hopefully this will work since so many developers tell about it. People have no idea how hard these things are to design. The proccess should be quite simple where I copy the stack to dynamic memory and set the A7 to the copied stack. When I want to restore the stack, I just set the A7 back to the original location and Jump to the right location in code. This way I always have the threads stack in a saveable location in dynamic memory. Cobalt doesn't allow to pass dynamic memory through SysAppLaunchRemote calls so I use dynamic memory to pass the stack but use feature memory to pass it back which means that on Cobalt it will be slower although the proccessor should be faster anyway so no big deal hopefully. Garnet on the other hand just passes a system dynamic allocated stack which is used and switched between application launches. Once I know that I can swatch threads locally, I will write PublicThread component which will allow the object storage to be able to multi-task (so you can copy an object and switch to another thread in between).