Continuing down the path of defining a BNF for the definition of a spiking neural network (SNN), layers with different properties could expand the capability of the SNN. For instance, an SNN with only spike time dependent potentiation (STDP) is not likely to have the capabilities of a network with both short and long term synaptic modification rules. Yet it could provide some flexibility to the network as a whole to have both types of synapse modification available simultaneously.

I’m going to outline a grammar based method for defining a neural network topology using multiple layers of Backus-Naur Form “grammars. The objective of this method is to provide a way to define the building blocks of a network and to attempt to find repeatable patterns that are useful.

I assume one type of neuron in a network with delays ranging from 1 to 20 milliseconds. More neuron types can be added later. I want to be able to define small polychronous groups of neurons that can be combined with copies of themselves and with other polychronous groups.

The reason for defining the network in terms of polychronous groups is that these groups can represent simple or complex calculations, memory units, etc. The polychronous group representation is basically an encoding of the synapses between a starting neuron position and other neurons such that activation of the initial neurons will cause activation of subsequent neurons if the synapse gains are high enough for activation to occur.

The next grammar layer is that of the network. The network is built by overlaying polychronous groups on top of each other with starting neuron offsets and rotation in the grid defined at the network. The polychronous groups defined at the lower grammatical layer define a map between points. The points define the neurons of the network. The delays between neurons are defined by providing a function of delay per unit distance between neurons. Thus by selection of the mapping by polychronous group, delays are inherently defined.

As you might guess, some neurons may be defined with some/no inputs or outputs. If there are no inputs, there is no point to keeping the neuron. If there are no outputs, and the outputs that exist are not used for network outputs, then those neurons could be pruned. If a polychronous group is placed such that some neurons would extend beyond the limits of the network, then the position of the neurons extending past would be wrapped.

Different neuron types could then be added by specifying different layers of neurons with the same grid as the first type. The polychronous group layer would then select the layer that is being connected for each neuron. If this is utilized, a meta-grammar could be defined which is weighted for the probability of selection that you desire.

More to come on this later.

I’m working slowly to implement Izhikevich’s neuron model in clojure. Real life is getting in the way, though, so it may take me a while. In the process, I’m trying to come up with syntactical simplifications to take advantage of the lispy language while maintaining good speed.

I’ve got to say, I kind of feel dirty dealing with the Java language at all (Clojure is built on top of Java)… but I’m dealing.

Age Related Population Structure (ALPS) is a nifty way to segregate different sections of an evolutionary algorithm. Using this method, you build in multiple sections that each have a maximum age limit. If a genotype reaches the age limit, it is checke to see if it is more fit than any individual in the next layer. The lowest layer is periodically refreshed with random initial conditions.

The stated reason for layers is to provide a method by which you can maintain genetic diversity in the population without getting stuck in a local optimum. I think you could get a similar type effect by creating fitness layered populations. Section off different members by fitness and only allow breeding at that layer or below. Of course, you could probably extend the concept even further by simply limiting individual breeding choices to individuals who have lower fitness and getting rid of layers entirely. I’d call this the ‘Popularity limited breeding method’. You can’t initiate breeding above your station.

Now, I’ve heard of hierarchical fitness layering, but I think there’s some synergy between the methods that could result in faster convergence on the global optimum. The multiple age layers allow for competition only with similar age solutions, but allows breeding with lower fitness individuals. Once an individual is too old for it’s layer, it either graduates to the next layer if it is fit enough, or it is retired. Another way of looking at this is that the rate of improvement of that strain of genetic material has leveled out enough that it isn’t worth exploring any more. So why not take the fitness layer concept and retire solutions when their average rate of improvement of fitness drops below their competition. Of course you’d want to keep your peak fitness individuals via elitism, but you shouldn’t need to keep all of the sub-optimal solutions when they loose their steam.

Any way, I haven’t tested my popularity contest method, so I have no idea of the merit.

My genetic programming tool is” now smart enough to do symbolic regression. That is a “basic” capability for GP tools. Now I’m going to work on improving it a bit so that it can find solutions faster and more generically. Functionality I’m looking for: automatic function generation, automatic looping and recursion, structure modifying changes, memory.

I’m building this programming program in Clojure, which is a newish dialect of Lisp implemented on top of Java. I’ve never used a Lisp before, so it is an exciting learning curve. I’m really getting into this whole functional programming kick.

Some guys needed to implement a Fortran equivalent function for a Simulink model I made yesterday… they wanted some initial code to start from, so I implemented it in an embedded Matlab block. I initially conceived the program in a functional style, then I realized that they needed to implement it in Fortran. Not the most functional-friendly language.  Oh well.

After using our first kitchenaid for a while to make some breads and pizza dough, we found that it was unable to hold up under the larger batches we wanted to make.  So Erin got me a professional 600 series Kitchenaid series.  We figured that since it was bigger with a more powerful motor that it would be able to stand up to some of the bigger or tougher doughs.

Well, one day I was making a batch of pizza dough which uses 7 cups of flour.  It’s enough to choke the old mixer, but I’ve made it several times before and hadn’t had problems.  This time, the motor started choking on the dough after it had been mixing for a few minutes.  It slowed down and just would not move anymore.  I removed the dough and tried again…it wouldn’t move even with no load.

I figured that the temperature protection switch had flipped and it would just start working again in a bit, so I waited another hour and tried again.  Still nothing.  Well, I’d already finished the pizza dough, so I put the mixer away to work on another day.

I pulled the top of the mixer off.  I found that I was unable to move the rotor, so I figured the gears may have been messed up.  I pulled out the four screws from the gearbox and pulled the top off.

All of the gears looked fine.  When I later pulled the motor out, I found that I could move all of the gears smoothly.  It turned out that the motor wasn’t turning at all.  Something inside it appeared to have fused or otherwise started binding.

After doing some electrical testing to make sure that the motor didn’t appear to have any shorts or opens, I decided to pull the motor out.  I labeled the four wires that wrap around the motor and connect to the control board.  Two of the wires come from the plug, and two connect to the motor.  I unscrewed the motor and pulled it out.  The gearbox needs to be open to remove the motor.

After removing the motor, I found that I was unable to turn the motor shaft, even with pliers.  Looking into the motor, I couldn’t see anything wrong.  The bearings also looked ok from the outside.  So I kept digging.  The motor has a bridge piece that braces the shaft right as it enters the gearbox.  That piece can be unscrewed on either side, but doesn’t immediately come off.  In order to get further into the motor, I had to remove the fan from the back end of the motor and hammer the shaft from the back.  Using a screw driver and rubber mallet, I hammered the rotor out of the motor.

It was being difficult to get out, even after I freed the shaft from the rear bearing.  It became obvious after a little inspection what the problem was.  Some of the insulation from the rotor had actually melted and flowed out of the rotor and solidified on the outside of the rotor.  This melted insulation caused the binding between the rotor and stator.

Between a flat-head screwdriver and a piece of sandpaper, I was able to remove most of the melted insulation.  The rotor fits very tightly into the motor, so any amount of melted insulation can be a problem.

When reassembling the motor, the brushes were a bit of a pain to get out of the way.  I used Erin’s help to pin them back with some screw drivers pushed through the back end of the motor.  There’s probably a much easier way to do it, maybe using paperclips bent properly, but this worked.

I reassembled the mixer and it worked at all speeds.  I have yet to make more dough, but I’m hoping that it will continue to work under a load.  I guess that the moral of this story is that your kitchenaid may not trip the heat protection switch early enough.  If you are making multiple batches of dough, give the mixer a break.  Even though it is labeled as professional grade, that doesn’t mean it can be run like a professional might run it.  If the mixer is having trouble turning to the point that the dough hook stops for more than a moment or two, it is at risk of melting insulation.

Skype was actually useful for this, Doug was able to see the problem and together we were able to figure out how to get everything apart.

Hopefully this helps someone.  There is very little information out there for the professional kitchenaid series.  Maybe someone else can also revive their mixer instead of converting it to a paperweight.

Erin and I were driving around last night and the window started getting a little fogged up. So Erin threw the defrost on and a bit of heat…however no heat came out. Doug and I decided to fix that today.

We found that the heater core was working fine…the output hose felt warm to the touch, so we figured we’d have to find out whether the heater control was disconnected somehow. We unscrewed the air flow control panel so that we could analyze the cable issues. The hot air control connects to an arm via a cable similarly to a bicycle brake cable. It turned out that the arm had broken where the cable attaches.

The fantastic part of this was that the arm was visible through the control panel and it was very obvious what was wrong. The not so fantastic part was that the opening was just big enough to get the arm in and just small enough and sharp enough to hurt each time. We pulled off the passenger side of the console to get a little more room.

In order to fix it, we drilled a hole further up on the arm and fashioned a bracket out of a folded 2 inch piece of sheet metal. Of course, the arm had a bit of a ridge where I had to drill the hole, so we had to cut that off with a dremel-type disk cutter. We got it cut and drilled a couple holes in the bracket. Getting the bracket attached to the arm was a real pain since I couldn’t actually hold the nut.

The cable sleeve was attached to a bracket with a little clip. I pulled the clip off and it sprung away somewhere into the bowels of the console. That sucked because we couldn’t find it and our little fix wouldn’t work without being able to secure the sleeve somehow. Long story slightly shorter…we made our own fix.

After closing everything up and figuring out if it all worked, we ended up with cuts and red marks up and down our arms and hands. It feels nice to be done.

On top of that, we reconnected the transmission pan since the gasket had failed and started to leak. That went about as smooth as possible and the car is driving great after 8 years.

Now that Erin and I are both wearing a boot, we tend to get some funny looks and comments while we are out and about. Lots of people just randomly ask how we both ended up with injured left feet. At least that is better than the people who just stare slack-jawed.

I am also feeling very priviledged and lucky to have a wife who likes to buy me tools. This weekend, Erin told me that she wanted to buy me a drill press for my birthday (Saturday). So we researched together and purchased a very nice Craftsmen Professional floor mounted drill press. This 17″ drill press is huge. It’s got 16 speeds, 3/4 hp, a nice table and a laser attachment. While I’m not fully convinced about the benefit of the laser, this is a heck of a tool and should prove useful in possible future projects.

Check out more pics in the workshop album pages 6 & 7.

I’ve (mostly) completed my fastest project yet today (With lots of help from Erin).  I’ve had my grinder sitting seldom used on a bookshelf.  Now it’s on a mobile platform with locking wheels.  I’m going to put a drawer in the box later.  The nice thing about this project is that apart from the wheels, I built it entirely from materials I already had.

The pic links to the workshop album.  Check page 6 for more photos.

I’m amazed at how fast the cabinet is going now.  The shelves and side panels took very little time to cut and fit in the cabinet.  The back and sides are done.  I’m already done with the keyboard door.  It folds out and is supported by some chains.  I just have to cut one more door for the printer shelf.

I suppose that one lesson learned on this project is that I should avoid mortice and tenon joints in pine.  Since the grain tends to crush instead of slice, it is difficult to get clean cuts with a chisel.  I should have spent the time to figure out an appropriate method with the router and router table so that the cuts were all cleaner.

Another problem is squareness, the cabinet is mostly square, but there are some minor gaps here and there that are more annoying than a problem.  I think that if I were to do this project again, I would not glue the entire carcass together until I had side panels ready to go.  If you can glue the carcass together and keep it square with the side panels, it would be easier than trying to figure out some way to square it with band clamps or pipe clamps.

With the way the project is shaping up, I ought to have it done within a year of starting it!  It all started last December, so I’ve only got a little while to go.