Smart Playlists

Nearly all the latest handhelds have music capabilities. Many of us have huge collections of music files on our PC. But with very small SD/MMC/MS cards there is not much room to store the vast amount of music we have available. So we manually move files back and forth which results in alot of wasted time. Many of us are just to lazy or frustrated to do this so we leave songs on our handhelds for even a month or two swapping just a few songs. An object storage could ease this by allowing to select files from your PC and allow them to 'swap' daily/weekly or even monthly on your handheld's capacity. However who has time to do this while at home/work, so why not do so on the way... have your entire music collection meta-data downloaded to your handheld and do the playlists from there (the meta-data is for sure smaller than the music itself). You could while on the bus create playlists, attach them to appointments (for start/end of the online range in which the playlists music should be on the device) and when you hotsync after the start of the appointment it will include the music files on the 'sync' and after the end of the appointment it will remove the files from the device making room for another playlist or other data. You can make the appointments recursive so that they continue to download once a month, week and so on. The user interface could make this functionality very usable by doing all the appointment work for you. If we took this a step further, we could say that all music with a high-rating are not rotated which means that if you like a specific track, just mark it high-rating and it will stay on the device even after the playlist's appointment has ended. The idea is that once the basic framework is implemented you can take this to boldy go where no one has gone before. I have asked many users and they dream for this feature. This is one of the reasons why I am pushing so hard on the object storage. It is crucial for the object storage to take these dreams into reality. This idea could be extended to other file types like documents. Wouldn't you like to not have to remember to download that presentation for next months meeting without downloading it now? I know many times that files that I wanted on my handheld were only valid on a specific date range. By not limiting the object type, it would allow to extend the same idea to any set of objects. This brings us one step closer to a world where it is not limited to 'NOW' but can remember tasks for the future. Isn't that one of the main reasons we have handhelds... NOT to forget future tasks... why cant it try and do some of them for us so that we dont even need to remember.

Update

Well... you are probably (if any one even reads this blog) thinking that this project is dead... oh well... for those who still have faith... here I go again: I will now code the core code seperate from the API. This allows the core to be more coded for the internal design than user easy coding and allows the API to be totally it's own design instead of sharing the core code. Since most of the core code is not even available in the API (Storage, Memory...) there should be less of a problem of designing for the user and can be more system oriented. The API also uses messages to send/receive functions while the core is internal and direct function calls. Note that the core is only the BASE of Screens Environment. Everything else uses the API so it gets functionality like inheritence and expansion qualities. I will be coding test code (to find API/Code problems) on my handheld daily while using the PC for actual releases. This allows me to find bugs before I get home since I have more time on the road (my job) then at home. I cant wait to finish the core code... you might think why it's taking so long? well... it's because the core gives me an easier start for the rest of the entire system. It removes the need of data structuring, classes (using run-time classes instead of code classes which allows system level inheritence and expansion plug-ins), memory allocation (virtual memory design), VFS file access and other file systems using plug-ins, window allocation and control manipulating and much more. The core is the 'system' which the entire Screens Environment design is based on. There is not a single component which does not benifit from the core design which is why it is so crucial to build such a design correctly. No other OS has such a system from the core for handhelds. When the betas are released, you might think that you waited to long for nothing but in very little time I can add features because of the core. My first version 2 years ago had loads of code behind it with just one line of code to display a window. This design has been implemented in the today version where little simple code calls loads of code which makes code more readable and debuggable.

Update

Well... What a week... My handheld hard-reset and corrupted my diary document (damn Documents To Go by DataViz) which means a few months of work down the drain and I now have to remember it all (or most of it). I have downloaded a new application for my handheld called 'Meteor' which has been very usefull so far in allowing me to structure my OS on the road. The storage and memory components have been merged into a single component called Data. It's been made as simple as possible with no stream IDs, just plain functions like new, free, read, write, size, resize, count, getbyindex and so on. I am still looking for users to help me via Windows Messanger (zhamilton1@yahoo.co.uk). So anyone at all (you dont need to be technical/programmer) who wants to help, send me a message. If you also want to help, you can post comments on each blog post you want to (by tapping on the comments number at the bottom of each blog post). Being married has realy helped the project, I am doing more work now then I ever did. I have at least a few hours a day I can concentrate on the more in depth problems. I am not coding blindly but catching every problem which flies my way while designing the API (it seems to catch most design problems). Once I have the data API coded it will be much easier to continue. Once the object API is coded then the party starts. So if you want to wait untill that stage... no problem ;)

My Plan

Many of you are probably reading through this site and adding it to a pile of existing either non-working or not bringing much to the table operating systems. This is not one of them and hopefully you will understand why... 1. I built no 'low-level' kernal I did not make a bootloader or any low-level kernal and dont plan to either. I am using the existing PalmOS platform as a base for my enviroment/OS. This gives me already some kind of streaming (PalmOS Databases), Display read/writie and Input (via pen/keys). This already takes me out of the dead ends most kernals have problems with and know the code works (I have a PalmOS handheld you know) and is improved over time by the company PalmSource (cobalt anyone?). 2. No emulating/simulating I am not going to support PocketPC software, nor will it be a Mac emulator or even a Windows/Linux simulator. Screens Environment will do one thing and that is run Screens written software (using the Screens API) when the Screens Environment is running. To PalmOS, Screens Environment is just like an application about as hacking as the PalmOS calculator = NONE! So you say what about PalmOS applications... although Screens Environment uses the 'Sub Launch' launch code it still needs the applications to register themselves before they can run any single call to the Screens Environment API. And even after that, no globals since sub launch does not support globals. Even if you know how to load your own globals - DONT for a simple reason... performance... An application can get more than 5 launches in a very small time period (like a few seconds) and that's talking about EACH application when multiple apps are working multi-tasking... which get's to my next part: 3. Cooperative Multi-tasking Screens Environement does not 'add' preemtive multi-tasking to PalmOS which includes scheduling also. Screens Environment includes a simple list which it adds application messages at the end and runs them (and removes them) from the start. This is how the current PalmOS works and I have no intension of messing with the internals of PalmOS to implement such a design. My design goals for this project are not imaginary and have been researched months to be made more and more concrete as time goes by. So I dont have it out yet... so what... I have been shaping this OS for more than one year (just to think about my design goals was a problem) and most of the second year (and still going) has been spent on just the CORE! What does the core include: 1. Storage This component takes multiple streams (like multiple dynamic allocations or multiple records of a database) and merge them into single long streams. This is to remove the known 64K length problem of untill today PalmOS versions (cobalt should relax the 64K length issue). 2. Memory Long streams are useless... so this component splits them into a list of allocations however with a twist. Data is stored from begginging to end of a stream while the header is stored in the opposite direction from end to the beggining. Both data and headers are added and not removed/moved untill they meet (basicly when you try to add data that will overwrite the header space) and then they are aligned. Aligning removes all freed allocations by moving allocations to be aligned one after each other (from begging to end for data and from end to beggining for headers) overwriting any free space. This means that data is hardly ever moved and aligning is done when you turn off the device anyway (dont worry... if you turn on your device in the middle... it stops aligning and continues the power-on and it doesn't take very long for a normal amount of time between power states, about a few seconds at maximum). 3. Object Now this is what is taking most of my time up... why? This is the object oriented storage system. Much like the future WinFS but on a smaller scale, it allows to store streams and structures of information in a single database with path and searching interactions. Supporting multiple classes and extensions per object, information can be 'typed' and 'extended' very easily. Each class and extension is an object in itself which causes a rich tree of paths. Each object also is a container and can hold objects or links to objects (links dont continue the structure). 4. Messages This provides the dynamic part of the object component using objects of type 'method' and doing inheritence/extensions when calling a method through an object (method objects are just stored as child members of an object). It also uses a 'messages' object to add/remove message objects that are posted via the message API. The entire programmer API is basicly made of message object postings. 6. Layers This uses dynamic objects (using the message API) for holding draw permatives like lines, rectangles, circles and so on which can be drawn on-screen at other times. All windows/controls use layers as thier underneath model for drawing to the display. Layers also provide transparency/translucency by using off-screen buffers. These are the core components that once coded, the rest of the system is basicly UI based features like multi-user support, window management and so on. This is not easy either but it's more easier to see it working forward than the core components. Also unlike the other components, the core components can hardly change once coded because they are hard-written code of function calls while using the object storage allows dynamic class management and extensions. If I write the window component to use only x & y coordinates, I could always extend the window component object with a z coordinate handler. This allows to always extend components without having to think much forward at all. Since extension handling is only run when you change an object's structure, most of these extensions go with hardly any speed performance loss at all (no I have not checked this... but using two message of two different handlers go through the same mechanizm as two messages for a single handler).

Double Design Direction

I am starting to feel like all these kernal programmer hobyists who feel like they are coding in the dark. This is not the way it's meant to be... I dont want to end up like all these 'dead' projects so I have decided to attack the problem directly. I will be coding from bottom to top (core to visual) like I have done untill now but at the same time code from top to bottom (visual to core). How is this possible? By seperating the graphics component from the object design it can be faster, bare-bones and allow me to make a 'mini-ui' to test the components that untill now have not been tested because I had nothing to test them on (walking in the dark). The graphics component (ScrDraw) and the text component (ScrText) will allow to draw directly to the display without any need of off-screen displays and the like. This seperation does not in any way remove the UI functionality that I want... actually it improves it by only making 'static' UI controls (buttons, lists and windows) using the layers component (based on the object component) while dynamic controls like progress bars, game views and animations using the draw component (no dependency). This will also allow me to finally give you all some visual progress on what's happening. I will able to post the 'testing' UI to show you all I am actually coding and not wasting my breath. How long? I have the system draw functions coded. I still have the text and generic functions like the mini-UI API but I should finish this by the end of the week. Then I can test the code of the first component (storage) and continue working on the memory component. I will publish the screenshots of the test UI as time goes by so just wait a bit more. I will be mixing between a CLI (Command line interface) and some graphical capabilities for the test user interface. This has nothing to do with the final UI and is for testing the core components only. If any of you have some ideas for the temporary UI post in the comments... I am always open for suggestions of any kind.

Memory Component

Good News... the API's have been locked (not modifyable untill the demo is released) and the first component has been coded. I am now working on the code of the second component. The second component is the Memory component. The memory component takes the stream that the first component (Storage) supports and breaks it into blocks. Each block is a memory allocation which is created in the stream so it keeps throughout application launches and soft resets. The header of each memory allocation is stored at the end of the storage stream. Header = StoreSize - 1 - (Index * HeaderSize) The '-1' is to get the last 'address' of the stream which is always '-1' because the address is zero-based. The number of header entries is stored in the first 4 bytes of the stream. The data of each header is stored between after the 4 bytes and the last header entry. The memory component works in an 'add-only' design. This means that if you remove a memory allocation the header data pointer is NULLed to specify that the entry is empty. No moving of allocations is done and new allocations are always allocated after the last allocation (if the last allocation was NULLed then the new allocation is written after the last 'valid' data). This allows for no moving at all of allocations in normal use. When does the system realign itself and gain all free space? When data is written and it will overrite a header entry (the data has reached the header part of the stream). All headers are compacted first and then it checks if it fits. If not it realigns all allocations. This might look like a large proccess but a few things remove the problems of this: 1. Data is only ever moved once in the realign. This is because everything is aligned from start to end. 2. Most realigning is done at the end of the stream since many of the allocations dont change thier location. 3. The Screens Environment automaticly aligns what it can when the device is turned off (mantience mode). This happens very quickly and even if you turn on your device in the middle, it stops aligning. This might sound a CPU kill but if implemented correctly the user might never actually see a single align message if the user turns his device off many times a day and the stream is big enough. The system notifies the user if the stream needs to be made bigger. If any of you have questions... post in the comments and I'll 'edit' the post to answer your questions. Hopefully I should finish coding the component quick so I can work on the object component (the hardest component of all).