Michael Sheldon's Development Stuff

Today I created a new STX LiveCD so I could test the new installer. It’s all coming along pretty nicely now, though I’m not done yet. A few little modifications I made to the basic STX install were:

• Changed the GTK theme to “Redmond” for more consistancy with FLTK programs.
• Recompiled PyFLTK so it used Xft to anti-alias fonts.
• Changed the desktop icon label font size from 10 to 12 to make it nicely anti-aliased.
• Added my patched version of eworkpanel for notification area support (still buggy, but better than nothing).
• Started looking at slapt-update-notifier; looks like I might need to hack EDE a little to make libnotify work though.
• Switched the X cursor to whiteglass since EDE seems to display the core cursor differently to everything else.
• Added a hamster to the corner ;).

I don’t know how many of these changes stibs will want to make use of though, so don’t get too attached to anything ;).

And here’s a pretty screenshot:

After much procrastination I’ve now tackled the installer backend. What had seemed like a mammoth task actually only took a couple of hours (though it is rather hackish in places). There’s still some work to do, a few bug fixes and post installation configuration stuff but it’s coming along fairly nicely now. Tomorrow I’ll grab a copy of the current installation script from RC3 and make it compatible with the new installer. I think stibs is planning on making the final 1.0 release pretty soon, so I expect the LiveCD installer won’t make it in until the next release.

I finally got around to finishing the design and custom content management system for a local band’s website. The CMS is written in Python and is an extension of the work I hacked together for the West Wales LUG a while back. It offers very easy WYSIWYG editing of pages and integration of gallery and planet. It’s currently available from http://midnight.mikeasoft.com.

There’s still a small problem in Firefox 1.5 due to a change in the way the body’s background attribute is handled. This stops the cityscape appearing at the bottom of the page as it does in previous versions and in all other browsers. Setting the height of the html element to 100% fixes this for pages shorter than the height of the window but breaks any longer ones.

Sometime soon I’ll try and package it all up in an easy to install form and clean up the administration tasks a little, just on the off chance that someone else might find it useful.

It would appear STX has had a rather favourable review over at tuxmachines.org. Hopefully I’ll be able to force myself to sit down over christmas and get the new LiveCD installer in to a working state.

We also had the developer behind SLAMPP showing an interest in our control centre and the new installer, so hopefully our tools can be of use to them :).

During a 2am perusal of Alex’s code we decided that watching gcc compiling stuff was rather soothing, as is listening to music. The potential for combining these two is clear, so I suggested an XMMS visualisation based on gcc. Today I hacked together a quick prototype.

The source can be downloaded from: http://junk.mikeasoft.com/xmms-gcc.tar.gz.

It’s still in need of rather a lot of fine tuning and needs extra compile phrases added to it, but it shows off the general concept nicely ;).

Following on from my post about the general design of AnNet I’m now going to have a go at outlining my ideas for the distributed peer-to-peer database that it’ll use for storing the DNS entries. Please take any literal values with a pinch of salt, these will probably change dramatically based upon testing and are likely to alter dependant on the size of the network.

The system makes use of multiple redundant meshes of nodes. Each mesh has 27 nodes, 26 store data and one acts as a co-ordinator. Each node is aware of the addresses of the 27 other nodes, the 26 in it’s own mesh and one link to another mesh. Each mesh contains two complete copies of the entire database, indexed by starting letter.

This allows for missing nodes without loss of data, giving time for newly connected nodes to replace missing ones with minimal global traffic.

This structure keeps all request traffic (which makes up the majority of traffic in the network) within local meshes with direct connections between nodes.

One node acts soley to co-ordinate the mesh. All data stored by the co-ordinator can be obtained by querying all nodes in the mesh. So the co-ordinator is easily replaced and acts purely as a bandwidth saving mechanism. The co-ordinator maintains data about the state of the local mesh and assigns tasks to new nodes (i.e. what data to retrieve and serve). Nodes speak directly to each other within their mesh, communication does not go through the co-ordinator.

Where possible a node will inform the co-ordinator when it is about to go offline, if a node makes a request and finds the node that should hold it is missing it will then report this to the co-ordinator and go on to ask the other member of the mesh that holds this data. Upon having a missing node reported the co-ordinator sends out requests to other meshes asking for any spare nodes.

Communication is carried out over UDP, so no continuous connections are maintained between the nodes.

In a future post I’ll cover the initial planned rules for handling the creation of new mesh groups and the propogation of changes throughout the network.

I think having domain registration controlled by either private business or the state is a bad thing. There’s an idea I’ve been toying with for nearly a year now, which is to construct a peer-to-peer based domain name system. I’ve now finally started hacking together a prototype (I had fiddled with a small http proxy based version last winter, but never finished anything serious).

I’m building it so it’ll integrate reasonably naturally with existing systems, it communicates using the DNS protocol specified in RFC 1035. This means that to the user .annet domains are just as functional as any other, they can run nslookup mikeasoft.annet send mail to mike@mikeasoft.annet or type http://www.mikeasoft.annet in to their browser and all will function as expected.

On the technical side what happens is quite different though. Each AnNet DNS client also acts as a server helping to host a massively distributed database of domain registrations. The domain name to resolve is passed through a series of hashing functions which then return possible ranges of nodes within the network that might hold the relevant data; the hashing functions take in to account the estimated size of the network, for example with a small network a larger proportion of data will be hosted by each node. A lot of redundancy is built in to the network so that many servers will have the required information, allowing the client to gain some reasonable level of certainty as to the authenticity of the response. Connection to the network is bootstrapped in a similar way to Gnutella, so the client needs to know about at least one other node on the network and from there learns about the state of the network in general.

Domains are registered to a public key (PGP based), and any requests to change a domain’s information must be sent signed with the corresponding private key, thus allowing people to change the status of their domains and have the changes propagate throughout the network with minimal chances of hijacking.

I am not making any provision for dispute resolution. Dispute resolution ostensibly tries to solve issues where someone registers a domain that someone else believes they have a “right” to, however I don’t believe that registries should act as arbiters of domain justice. People can persue their trademarks through the legal system and have the state force one party to reassign ownership of their domain if they wish, but AnNet won’t help them circumvent the owner. So if someone’s stubborn enough they can still keep their domain at the expense of fines and even possible imprisonment (unlikely, but people make ethical stands about seemingly odd things sometimes). It’s purely first come, first serve as far as the system is concerned.

However, I am considering possible safeguards put in to place to revoke any domains registered to a specific public key if that key attempts to flood the network with registrations. I’m still not sure about this.

The prototype is still in very early development, it’s only had a couple of days worth of hacking on it (most of which consisted of me puzzling over the DNS protocol), but I thought it’d be nice to stick a reasonably detailed post up about my plans so people can criticise the impracticality of it all ;).

Finally sat down and got some more work on the STX installer done. Most of the GUI is pretty much finished, I’m now working on the flashy backend system. With the old SphinxOS installer the install instructions were coded in to the installer itself, for the STX installer I thought it’d be nice to try and separate them out a bit. Originally I had planned to have the instructions stored in an XML file, with a format something along the lines of:

<installer>
    <item>
        <title>Copying /var</title>
        <progress>23</progress>
        <command>cp -r /var /mnt/$ROOT</command>
    <item>
<installer>

But I came to the conclusion that this wouldn’t be workable as the installer would need support for certain conditionals (such as if the user wants a swap partition, or if the user wants to have home on the same partition as root). I then decided to look in to the use of the shlex python module (which offers simple lexical analysis features) with a view to creating a mini scripting language for the installer. Finally I decided it’d be very cool if I implemented a subset of bash’s scripting features, allowing the installer instructions to double up as a textual installer themselves.

Here’s a quick example file to show what I’m getting at: http://junk.mikeasoft.com/stxinstall.txt

Still a lot of work to do in actually implementing it, but I think it’s a fun enough solution that I might actually stick with it for a few days.

I’ve added fullscreen anti-aliasing support to SabreGL, unfortunately with my current setup I can’t actually test it. I’ve also been doing more work integrating the new 3DS code so animation speeds are handled now. I still need to sort out interpolation for the 3DS models (already works fine for MD2 models). Also made some changes to take more notice of specular and diffuse information in 3DS materials (not actual specular maps yet though).

I’ve gone one better than a screenshot this time, you can now see a video of an animated 3DS model.

After a few days hard work I’ve now rewritten SabreGL‘s 3DS model handling code to use the very useful (but badly documented) Lib3DS. So we now have support for animated 3DS models with vertex normals calculated based upon smoothing groups. It should also be trivial to add bump map and specular map support to the 3DS models once the engine supports them (which isn’t so trivial ;)). I still need to rewrite some of the collision detection code but this shouldn’t be a massive task. I’m now seriously beginning to consider the possibility of a release version (presently it’s only buildable from CVS), and Alasdair‘s kindly offered to have a go at getting a more up to date Windows version built.

Now for the ever so pretty screenshots: