What's Wrong with the Web

• IE6? PNG transparency doesn't work right.
• CSS is a complete mess of a system, which doesn't specify how to handle a whole variety of different issues correctly. That'd be things like:
• Scrollbars in overflow blocks
• Centering of elements without resorting to margin tricks.
• Z-Order handling (I mean, come on, a z-index? That's a little retarded).
• Things which should be easy currently require at least a day of trial and error to get right (I mainly blame CSS for this)
• Fonts can't be downloaded. Still. After 14 years. (Without browser-specific extensions).
• CSS doesn't separate layout from the document any better than HTML did.

How do we fix it?

I want to see a new way of building web pages that completely ditches the old crap. Here's what I want to see:

A Tic-Tac-Toe style Layout Model

Every block should consume space from its parent using a tic-tac-toe grid type of arrangement. You should be able to specify which elements in that grid can stretch, and which ones are corner pieces which don't. And the center should fit the content.

Of course, we'd need ways to break out of this system and absolutely position elements - but this will handle the 99% case of the layouts people put together.

We'd also want a "flow-around" algorithm that lets you flow text around these elements, regardless of the layer they're on.

Element Grouping

Once you've built an element out of component pieces (like, for example, a nice button which stretches to fit the text on it, and has a drop shadow, etc), that element should be groupable. You can then modify the location of the entire group. In terms of rendering and mouse-handling, the group would render as a single unit.

You could also group groups together. Think of the way that most vector illustration programs work.

Element Templating

Once you've built an element out of pieces, you use a template to refer to it on your page. For example, if we have a Panel with a title, and a content block, and an associated sidebar, it'd look something like this:

<element class='panel'>
    <template name='title'>My <em>Cool</em> Panel!</title>
    <template name='content'>My content.<br>
        <sidebar>Templatized sidebar element goes here</sidebar>
    </template>
</element>

... and you'd only define it once. So if you wanted to have a web-part interface, you'd just have one of these which had the <element class='webpart'></element>, and fill it with whatever you want - and all of the associated behavior of the webpart, and the custom rendering chrome for it would come with it. And you would only have to define the webpart once. Which would be great for skinning in a logical fashion.

Where the templating idea becomes even more interesting is when you start using a content map with the templates. For example:

<content>
<tf id='SarahPalinStory'><tf id='headline'>BREAKING NEWS: Sarah Palin - part woman, part fish!</tf>
<tf id='firstpart'>This just in - Sarah Palin is part woman!</tf>
<tf id='sidebar'>Sidebar headline!</tf>
<tf id='mainpart'>Main text of the article goes here</tf>
</tf>
</content>

So that's your content page. It's layout free.

Now we need a layout to go with it:

<element type='newsarticle'>
   <template name=headline contentsrc='SarahPalinStory.headline]' />
   <template name=firstpart contentsrc='SarahPalinStory.firstpart' />
   <template name=sidebar contentsrc='SarahPalinStory.sidebar' />
   <template name=mainpart contentsrc='SarahPalinStory.mainpart' />
</element>

Needless to say, we could just leave the contentsrc parts off, and seek through the content based on the name, which would give us something like this:

   <element type='newsarticle' contentsrc='SarahPalinStory'>
      <template name=headline />
      <template name=firstpart />
      <template name=sidebar />
      <template name=mainpart />
   </element>

... and if we want to go a step further, we can default to filling all of the element's template values from the contentsrc, in which case our definition collapses to this:

   <element type='newsarticle' contentsrc='SarahPalinStory' />

Of course, we could also use a separate web page for the content - eg.:
   <element type='newsarticle' contentsrc='http://newsfeed.com/SarahPalinStory.cht' />

... and we could even have, on the same page, a list of current stories, which we find the contents for by index, inserting the appropriate elements in a loop.

The other handy thing is, once you have the text separated out, it's easy to define a text-flow. All you do is use the same content src in subsequent elements, and the browser should be able to layout the text between the blocks, making multi-column layouts a breeze.

Content Map Semantics

Content maps start getting important in other ways once you start considering the semantic web. For example, with our story above, you can easily produce a view of the site for blind people. Throw in a layer that lets you transform the content map, and you end up with a way to build a wiki and format it however you want.

You even get downlevel support for free. Want a simple version of the site that works on mobile devices? Strip out all the flashy stuff, and create a simple template which handles it.

A few Notes

Look, I'm not saying that this system as laid out is perfect. It's not fully fleshed out for a start, and would require a few months of hard thinking by a talented team to get there. But it's certainly a lot better than the blind alley we've ended up going down with HTML. Let's fix it. Soon.

Oh, and as far as the spec goes: The spec should be LAW. It should be reviewed by all implementors, and refined until there is no room for misinterpretation. Too many things are left to chance in the current specs.

One more thing: The W3C? Useless at this point. They're working way too slow to be of help to anyone, and they insist on carrying on down the horrible path they're on instead of fixing what's busted.

XNA SpriteBatch Stalling every Second

Public Service Announcement:

If you're doing simple 2D sprite stuff in XNA, and every second or so, the GC is hitching, it's because of SpriteBatch. Every call to SpriteBatch creates a new object, which eventually needs to be GC'd. This will give you a stall every second or so, as the GC runs and tries to clean up.

Kind of sucks really. Ultimately, it means that if you want to get around this, you're going to need to write your own Sprite rendering code that allocates everything up front, and doesn't release anything until a level change, and recycles as much as possible. Not too hard - just build a quad, and write the appropriate code to render it. But it's a lot more work than just slotting it in, unfortunately.

Math Puzzle: this one goes to 1.0

Your mission:
You have the following piece of code:

int GetIndex()
{
float rand = GetRandomNumber();
return NUM_ITEMS * rand;
}

... which you'd expect to return a value from 0 to NUM_ITEMS-1, giving a nice equal chance of picking any item from some list at random.

The question is this:

Most random number functions return a value from 0.0 to 0.999999999999999'. (Or, to look at it another way, 1-epsilon, where epsilon is the smallest possible floating point value that can be represented).

However, Joey Badrand (that scandalous cur) has come up with a random number function for your use which returns a value between 0.0 and 1.0! And you have no choice but to use it in your function.

Questions

1. What does this do to your distribution, and why is it a bad thing?
2. How do you fix your function so that the output of Joey's random number generator can be used to do the Right Thing?

Touched...

A while back I wrote an article on CodeProject about a bi-partite circular buffer algorithm I came up with to handle asynchronous network IO. (It's also useful for other things - pretty much any scenario where you have to pass data in contiguous blocks to other APIs, yet you don't know exactly how much data you're going to be passing at any time).

Well.. I just came across this blog post from someone:

A Pure Programmer
I read an article about circular buffer and related code, which written by Simon Cooke. It’s very good. I never heard of Simon Cooke before. I was moved by the last words in his article: “If you do find it (the code he write) useful, or use it in any of your code, all that I ask in return is that you drop me an email and let me know how the code is being used. It’s nice to know that it's out there, alive, and doing cool things."
Oh,what a pure programmer!

Awww... bless. It warms my heart that someone appreciates my work, it really does.

I must admit, I was rather proud of that little piece of code. It worked out to be pretty fast too - the only way to do anything faster would have been to use the virtual memory mirroring technique I laid out in the article - but unfortunately, that doesn't work on some architectures. (I'm looking at YOU, XBOX 360). Not sure if it's multiproc safe on other systems too. Damn cache coherency, I stab at thee.

Followup:

So I got a bit narcissistic and did a Google search on the Bip Buffer... and lo and behold, people are using it. One guy's looking at it as a way of performing least-wear writing to flash memory (now that's a cool application I never even thought of!). So glad this code's getting some use!!!

Debugging By ESP...

I had one of the strangest experiences I've had in a long time today. Almost up there with that big switch connected to nothing that when you turn it off crashes the computer, that you hear about in all those old hacker tales...

We've had a bug that's been really tricky to find and hunt down at work. On and off, it has caused lots of the monsters and other animated meshes in the game to vanish in a peculiar way, seemingly completely at random. This has been a problem for a while now, on and off, for at least 4 months if not longer. It's so intermittent though, that it's really tough to figure out what's going on.

Well, yesterday one of the developers finally managed to track down a repro case. After playing around with it for a while, I started getting a hunch that it might have something to do with how we do the lighting on the PS2 version of the game. Messed around with some of our debugging flags, and it seemed to corroborate it; depending on where you placed the camera in the scene, and depending on which lights were visible, bodies and heads would pop into and out of view.

I figured it couldn't hurt to take a look and see if I could track it down. A complete stab in the dark, but I'm a pretty good debugger, so why not?

So I went back to my desk, fired up Visual Studio, and loaded up the game engine code.

The project opened with the exact file the bug was in on top. The correct function was on screen. And my cursor was sitting on the line of code that was wrong.

Needless to say, this was a bit shocking. I stared at the code for a bit, then a bit more, and then on a hunch decided to check and see if the code was causing a problem.

The basic nature of the bug is this:

class RenderState {

enum BufferIndex {
PointLightBufferID,
PointLightCount,
DirectionalLightBufferID,
DirectionalLightCount,
SpotLightBufferID,
SpotLightCount,
MaxBufferIndex = 8
};

unsigned char m_LastLightingPass[MaxBufferIndex];
unsigned char m_CurrentLightingPass[MaxBufferIndex];

void SetupLightingPass()
{
if (*((unsigned long*)m_LastLightingPass) == *((unsigned long*)m_CurrentLightingPass))
return;
*((unsigned long*)m_LastLightingPass) = *((unsigned long*)m_CurrentLightingPass);
MoreSetupForLightingPass();
}

Ok, so this is a bit on the hokey side, but it's only for illustrative purposes. The code does a comparison of the two lighting state buffers, and a copy of one over the other if they're different (in which case it'll also do more setup work). For speed purposes, it does the comparison in 4-byte chunks, instead of a byte at a time. Which is great, because that's the width of the bus of the CPU, so it's really happy to do that comparison that way. It's nice and efficient.

Unfortunately, there's a bug here. The problem is that at some point, spotlights were added, but the optimization wasn't updated to take it into account. This is a pretty easy and understandable mistake to make, but it's a bugger to actually find.

(In case you're wondering, probably the easiest way to fix it is like this:

if (((unsigned long*)m_LastLightingPass)[0] == ((unsigned long*)m_CurrentLightingPass)[0] && ((unsigned long*)m_LastLightingPass)[1] == ((unsigned long*)m_CurrentLightingPass)[1])
return;
((unsigned long*)m_LastLightingPass)[0] = ((unsigned long*)m_CurrentLightingPass)[0];
((unsigned long*)m_LastLightingPass)[1] = ((unsigned long*)m_CurrentLightingPass)[1];

But unfortunately that doesn't get around the problem that if the code changes again, the same problem could still arise. With a smart compiler, one could expect it to make appropriate optimizations for us. Maybe hint to it that the data is aligned to a 4-byte boundary, and the default copy and comparison operators would be set appropriately. But we're not necessarily using a smart compiler to compile the code, so these kinds of things need to be added by hand.

(For the record, like most software studios, we're using a branched version of gcc for the PS2 builds).

Visual Studio, though, lets you make those kinds of declarations in code - I've not checked if it writes out appropriately optimized assembly code though. I'm willing to bet that it does.

Anyway... this is all getting away from the point. The point is... my IDE opened to the exact line of code that was faulty. All it took was for me to look at it, and, well, follow my nose.

Needless to say, the chances of finding this bug was like finding a needle in a haystack. (And, for the record, it still remains to be seen whether or not it actually fixes the problem). For quite some time after finding it, I actually felt completely buzzed, energised, and more than a little weirded out. I even felt kind of faint (although that may have been a mixture of lack of food and my nicotine patch wearing off). How on earth could a computer do this for me?

Must have been my lucky day. Either way, it's freakin' weird. Maybe I'd seen the code, like an idiot savant. Maybe it was subliminal! Or perhaps there's a ghost that visits coders who are clutching at straws! Maybe I was meant to find that bug...

... and now, the scientist in me takes over, and has to find out what really happened.

Well, let's look at the evidence - the IDE opens whatever files I had open last time I was in the project. So I must have had that file open. And it opens scrolled to the last place that it was open... so I must have been looking at that function.

I got all the way home from work before I figured it out. Yesterday, I'd been coordinating some changes to the way we render lights on animated meshes, and there are two files in which the lighting coefficients get set up. One of them - you guessed it - was this file. And, yep, guessed again, it's in that function too. So before I filed the task for one of the team to work on, I did my usual and investigated it and what it would require. After finding out from one of the engine programmers the appropriate places to make the changes we needed, I opened up those files to verify that everything was in order, and that it wasn't a tremendously risky change.

... and just by chance, the bug was in the same function as the last place I was checking.

So, nope, not ESP. Not even anything weird - just a completely random coincidence. Serendipity, if you will.

But I still think that I may have burned through some of the karma I've built up for a while for this one. I'd better not walk under any ladders for a while :)