TinyCPU -- My attempt at a processor

Well, one of my life goals (according to my CV) is to create my own processor in an FPGA.

Well, I've finally set out to accomplish this goal. So far, VHDL has been very mind expanding. Things happen sequentially, yet at the same time. Wrapping your head around it can probably be compared to understanding recursion in programming terms.

But anyway, I've finally got to where I can synthesize my CPU design and put it in an FPGA, load it's internal memory from an mbed/arduino/whatever and finally execute a stream of instructions... Of course it's not done, the only instruction supported right now is mov reg, immediate.. but it's a start. There is still a ton of work to be done with it, but hopefully it should be halfway usable as a processor in the next week or two.

If you're curious about the source code of my TinyCPU project, the code is at opencores

For reference, TinyCPU has the following goals:

  1. Mostly single cycle instructions
  2. Von Neumann architecture (code is data and vice-versa)
  3. 8-bit processor (8 bit registers and operations)
  4. 16-bit address bus (capable of accessing it by using segment registers similar to the 8086)
  5. Be simple code and it has to fit within my 250K gate Spartan 3E
Posted: 05/17/2012 00:48:00

But how does it work?!

I was going to put a rant in here about PHP sucking, but instead I'm going to do something useful and informative. Describe how this blog works.

When I started this blog, I had only used ASP.Net with WebForms. Even though I didn't know anyway to do it better, I could see the shortcomings of pretending HTTP is a stateful protocol. This is because before ASP.Net, I used PHP, which actually taught me quite a bit about the HTTP protocol. I've since abandoned the PHP ship due to stuff like this, but that's another story.

I believe at the time ASP.Net MVC was still pre-release or still filled with bugs, especially for Mono. I was set on using Linode for my hosting because I've never had any problems with them, and I prefer managing Linux servers to Windows... So I had a choice, Use WebForms, or use a different framework. I really like C# though, so I instead created a very minimalistic framework.

This framework eventually got broken up into 3 different pieces of software.

  1. EView -- a powerful, yet simple, static view generator written completely in T4
  2. EFramework -- a routing engine and basic replacement for using ASP.Net MVC or WebForms
  3. FSCAuth --a very easy to use Authentication library

At first, these were all one framework named EFramework, but I since broke them up since they are all useful by themselves as well.

So first up, I'll explain how EView generates everything you see here.

EView is just a single T4 file. I would've broken it up into separate files but at the time Mono had a bug that prevented me from doing it easily. How it works basically is you place EView into a directory, and when you run the T4 file template, it'll read all the files from the current directory and generate a C# file.

So, let's do a simple example. You have a folder named Templates and 3 files, EView.tt (the T4 file), IndexView.html, EntryView.html

So, you run EView.tt. Visual Studio (or MonoDevelop or whatever) runs the T4 template during design time. EView will scan the folder it's placed in. It finds IndexView.html and EntryView.html

Let's say IndexView.html contains this:

<!DOCTYPE HTML>
{@
Entry as IEView;
@}
<html>
<head>
    <title>Test Index</title>
</head>
<body>
  <div class="entry">
    {=Entry=}
  </div>
</body>
</html>

Now let's explain. What's that weird {@ thing at line 2? This is an EView directive. EView has blocks like { } The symbol after the { and before the } indicates what the directive is. The @ directive is for variable declarations. So basically, in the view class it generates, it'll add the public variable Entry of type IEView to it. IEView is the interface that all EView views implement. After that it looks like plain HTML until {=Entry=}. The {= directive tells EView "put this variable's value here. So it wants to put theEntryvariable there. EView is smart enough and knows that Entry is a View, so it callsEViewRender at that point.

So basically, this all generates a view class like this: (more complicated than this, but to be easy to understand, this example is simpler)

  public class IndexView : IEview
  {
    public IEView Entry;
    public string EViewRender(){
      StringBuilder sb=new StringBuilder();
      sb.Append(@"""<!DOCTYPE HTML>
      <html>
    <head>
        <title>Test Index</title>
    </head>
    <body>
      <div class=""entry"">
     """);
     //insert {=Entry=}
     sb.Append(Entry.EViewRender); //a lot more logic goes into this, but just for example sake
     sb.Append(@"""
      </div>
    </body>
      </html>
      ");
      return sb.ToString();
    }
  }

And that's basically how it works. So when you want to use IndexView, you just do this:

var index=new IndexView();
index.Entry=MyEntryView; 
string html=index.EViewRender();

So as you can see, it's very powerful really, while views are really simple to create. No weird XML tags to use, or other crao. You just write your HTML and use a { } blocks to describe how the view should be dynamic. It supports many more operations than shown here such as if statements, foreach statements, and a special Layout directive that works similar to ASP.Net's MasterPages.

So now you understand a bit of what my views look like and how they're rendered, we can move on to EFramework.

EFramework was really an experiment to see if what I was trying to do was even possible.. and then I thought it worked really well, so I kept at it.

What exactly does EFramework provides? It provides 2 things:

  1. A powerful routing engine
  2. An HttpHandler class that functions similar to ASP.Net's .ashx Handlers.

So let's start with the routing engine. The routing engine has 3 pattern matching engines

  1. Regex for regular expressions
  2. Plain for exact plain text matches
  3. Simple, used for my own pattern matching that resembles ASP.Net MVC's routing URLs

I'll only cover the Simple type of pattern matching.

Here is the Routing record for the pages of my Blog:

Routing.AddRoute("paged_index",PatternTypes.Simple,"/blog/{page}",()=>{return new BlogHandler();});
  1. "paged_index" is the name of the route, which is passed to the HttpHandler that handles the route.
  2. PatternTypes.Simple is which type of pattern matching
  3. "/blog/{page}" is the pattern to match. It basically says match anything that looks like /blog/* where * is anything, and put the wildcard value into the page route variable
  4. This is a lambda for creating a new HttpHandler. So basically, when a route matches, it will call on this Lambda and expects for the lambda to return an HttpHandler of some sort

The Simple pattern matching is also quite powerful and can do more than simple wildcards. It's aware of how / affects URLs and can do the following:

  1. "/blog/{entryid}/comment/add" This will match anything like /blog/*/comment/add where * can be basically anything but / and when it reaches a / it knows the wildcard is over
  2. "/test3/{id}/{action=[view,edit]}/{*}" This one is a bit more complex. The first {id} of course matches anything up til the next /. {action=[view,edit]} indicates that this wildcard can only be view or edit. If it is not one of those two, the route doesn't match to the URL. And finally, the {*} adds support for "descriptive" URLs, such as from StackOverflow: http://stackoverflow.com/questions/2185781/why-cant-c-sharp-compiler-follow-all-code-paths-through-a-switch-statement

That's about as complex as it gets right now. I'm sure there are some URLs it can't accept, but it works for most URLs I've seen.

Now then, how does an HttpHandler work exactly?

public class BlogHandler : HttpHandler
{
  public override void Get ()
  {
    if(RouteID=="page"){
      var v=new IndexView();
      v.Entry=GetEntry(RouteParams["id"]);
      Write(v.EViewRender());
      return;   
  }
}

Pretty simple. The Get function is called when the HTTP method is GET, Post for POST and etc. RouteParams is a dictionary with all of the route variables in it. So in this case, the route probably looks like

Routing.AddRoute("page",PatternTypes.Simple,"/blog/{id}",()=>{return new BlogHandler();});

And that's for the most part, what makes EFramework good enough for me.

And then there is FSCAuth, what prevents all of you from posting articles on my blog.

FSCAuth was made because I think ASP.Net's authentication mechanisms are a bit too complex for simple authentication scenarios, such as this blog. And FSCAuth is so easy to use and so secure(by default) that I decided to try to sell it. So how does FSCAuth tie into this site?

Well, I use MongoDB for my database (also mainly for learning something new :) ). FSCAuth works by having a UserStore class and a few configuration options set in Global.asax

First the easy part. My configuration looks like this:

        Authentication.UserStore=new MongoUserStore(); //set the UserStore to be used
        Authentication.SiteName="Last Year's Wishes"; //put into login cookies
        Authentication.LoginPage="/login"; //where to redirect to when authentication is required
        Authentication.UniqueHash=Config.UniqueHash; //our secret hash
        Authentication.CookieHttpOnly=true; //Set HttpOnly property on cookies
        Authentication.CookieSecure=false; //set to choose to only transport cookies over HTTPS 
        Authentication.CookieUseBase=true; //hash the physical path of the application into cookies. 
        Authentication.CookieUseBrowserInfo=false; //hash in client's browser information to the cookies. 
        Authentication.CookieUseIP=true; //Hash in the client's IP address to the cookie. Ties the cookie to the client's IP 

Now the last configuration thing to do is create our UserStore. This is a direct rip of mine (to show how easy it is to create one)

public class MongoUserData : UserData{
    [BsonId]
    public ObjectId ID{get;set;}
}

public class MongoUserStore : IUserStore
{
    public MongoUserStore ()
    {
    }
    public UserData GetUserByName (string username)
    {
        var db=Config.GetDB(); //gets our MongoDB database instance
        var u=db.GetCollection<MongoUserData>("users").FindOneAs<MongoUserData>(Query.EQ("Username",username));
        return u;
    }

    public bool UpdateUserByID(UserData user)
    {
        var userdata=(MongoUserData)user;
        var db=Config.GetDB();
        var u=db.GetCollection<MongoUserData>("users");
        if(u.Save<MongoUserData>(userdata)==null){
            return false;
        }else{
            return true;
        }
    }

    public bool AddUser (UserData user)
    {
        var userdata=(MongoUserData)user;
        var db=Config.GetDB();
        var u=db.GetCollection<MongoUserData>("users");
        if(u.Find(Query.EQ("Username",userdata.Username)).Count()!=0){
            return false;
        }
        u.Insert<MongoUserData>(userdata);
        userdata.UniqueID=userdata.ID.ToString();
        u.Save<MongoUserData>(userdata);
        return userdata!=null;
    }
    public bool DeleteUserByID(UserData user){
        var userdata=(MongoUserData)user;
        var db=Config.GetDB();
        var u=db.GetCollection<MongoUserData>("users");
        u.Remove(Query.EQ("_id",userdata.ID));
        return true;
    }
}

As you can see, it's not too complex to create... unlike a similar thing for tying ASP.Net's Forms Authentication to MongoDB.. I know it can't be done in only 58 lines of code anyway.

So then, how to actually make use of FSCAuth, all I do is call Authentication.RequiresLogin() or one of the other Authentication functions.

This is my actual Post method for my BlogHandler:

    public override void Post(){
        if(RouteID=="new"){
            Authentication.RequiresLogin();
            var c=Config.GetDB().GetCollection<BlogEntryData>("entries");
            var entry=new BlogEntryData{
                Title=Form["Title"],
                Text=Form["Text"],
                Publish=Form["Publish"]!=null,
                Posted=DateTime.Now,
                Tags=new List<string>(Form["Tags"].Split(new char[]{' '},StringSplitOptions.RemoveEmptyEntries))
            };
            c.Save<BlogEntryData>(entry);
            Response.Redirect("/blog/view/"+entry.ID.ToString());
        }
        if(RouteID=="edit"){
            Authentication.RequiresLogin();
            var c=Config.GetDB().GetCollection<BlogEntryData>("entries");
            var entry=c.FindOneByIdAs<BlogEntryData>(new ObjectId(RouteParams["id"]));
            entry.Title=Form["Title"];
            entry.Text=Form["Text"];
            bool p=Form["Publish"]!=null;
            if(!entry.Publish && p){ //if not previously published, update the time so it's "bumped"
                entry.Posted=DateTime.Now;
            }
            entry.Publish=p;
            entry.Edited=DateTime.Now;
            entry.Tags=new List<string>(Form["Tags"].Split(new char[]{' '},StringSplitOptions.RemoveEmptyEntries));
            c.Save<BlogEntryData>(entry);
            Response.Redirect("/blog/view/"+entry.ID.ToString());
        }
    }

And that's all I have to do. Super simple.

Even though I created this framework as a workaround because MVC at the time was very experimental with Mono, I found that I rather like my framework. I can see quite a few limitations, but it works wonderfully for small scale websites. I also have not measured the performance of any of this except for FSCAuth, so I'm sure it's not enterprise ready, but for this low-traffic blog, it works really well. And I learned a lot about ASP.Net in building it.

Anyway, hope you enjoyed reading how this site works and maybe are curious about one of my 3 projects. In case you are, here are the links to the projects again:

  1. EView -- a powerful, yet simple, static view generator written completely in T4
  2. EFramework -- a routing engine and basic replacement for using ASP.Net MVC or WebForms
  3. FSCAuth --a very easy to use Authentication library
Posted: 05/03/2012 20:16:16

VHDL Coding

It's been about 2 weeks since I set out to conquer VHDL and I think I'm finally to the point to where I can say I'm familiar with VHDL. Still learning a lot with each piece of code though.

Basically the learning curve of VHDL is so steep not because of VHDL as a language, but rather because describing hardware is much different from creating software. There is no real easy way to get over the learning curve either without persistence and practice.

So anyway, now as a learning project I'm making a super tiny CPU for my FPGA. I both hate and love VHDL. There is a ton of boiler plate code it seems like, and yet it's so powerful and expressive in describing hardware.

For instance, my simple register file for my CPU totals up to only 30 lines of code... and now imagine describing such a circuit with just logic gates as a circuit drawing:

library IEEE;
use IEEE.STD_LOGIC_1164.ALL;
use IEEE.NUMERIC_STD.ALL;
use ieee.std_logic_unsigned.all; 

entity registerfile is
  port(
    Write:in std_logic_vector(7 downto 0); --what should be put into the write register
    SelRead:in std_logic_vector(2 downto 0); --select which register to read
    SelWrite:in std_logic_vector(2 downto 0); --select which register to write
    UseWrite:in std_logic; --if the register should actually be written to
    Clock:in std_logic;
    Read:out std_logic_vector(7 downto 0) --register to be read output
  );
end registerfile;

architecture Behavioral of registerfile is
  type registerstype is array(0 to 7) of std_logic_vector(7 downto 0);
  signal registers: registerstype;
begin
  writereg: process(Write, SelWrite, UseWrite, Clock)
  begin
    if(UseWrite='1') then
      if(rising_edge(clock)) then
    registers(conv_integer(SelWrite)) <= Write;
      end if;
    end if;
  end process;
  Read <= registers(conv_integer(SelRead));
end Behavioral;

Honestly, it might should be more complicated than I made it, but I can already tell describing hardware in VHDL is going to be much easier than describing it in kcircuit.

Also, I actually have found that following TDD principles makes a lot of sense when describing hardware. Writing tests first shows you quickly how many possible ways your component will be used and allows you to develop and make sure it satisfies all of your needs.

I've never wrote tests first in software, but in hardware it actually seems more intuitive than writing them later.

Posted: 04/29/2012 02:44:22

Recent Projects

Being unemployed really sucks because all of the days kind of just turn into a blur.. but, on the bright side I have a ton of time for side projects. So coming up are a full blog entry(ies) over these two topics:

First, my FM transmitter built from an Arduino, an NS73M, a 4-digit serial 7-segment display, a salvaged car USB charger, 3 push buttons, and a few odd capacitors and resistors. Parts ended up costing me $40, took me about a week to code up my "just-get-this-to-freaking-work-so-I-have-music" code. It's still not done, but it's functional enough for me and I never have a static problem with it like most of the FM transmitters you'll buy at Wal-Mart. The only big thing I need to do now is refactor my code and design a (my first) PCB for it.

Second, hopefully some kind of VHDL project. I finally got done reading Free Range VHDL. Very good book as an introduction to VHDL and I got WebPack installed on my computer again for my FPGA. So hopefully, I'll have a tiny 4-bit processor or something built in a few weeks.

Anyway, I'll get around to actually documenting what I've been doing in a few days hopefully.

Posted: 04/23/2012 20:37:07

Building the Arduino Libraries from the command line

If you're like me, (few probably are) then you're one of those that likes long-term solutions instead of short-term workarounds.

Well, I've been building an FM transmitter using an Ardunio(more details about that later). When I first went to write code for my Ardunio, I noticed that my libraries are massively out of date. So out of date that most of the Ardunio code I found would no longer work for me. So, it was time to upgrade. I don't remember how I got the Arduino library before, so I set out to download a copy of it...

After an hour of searching... hmmm no one hosts a copy of it for some reason. Well, I guess I'll build it. I look at the ill-documented mess that is the Arduino source tree... no makefile for the library. The common solution is to run the Arduino IDE once, build a sketch, and copy the library from there. Well, that's not good enough for me. So I made a makefile and set out to get it to build from the command line

Directions:

  1. Download a copy of the Arduino IDE source code
  2. Copy the contents of hardware/arduino/cores/arduino to a new directory I'll refer to as arduino_build
  3. Copy the pins_arduino.h file from whichever Arduino variant is yours from hardware/arduino/variants (check boards.txt if you're not sure) to arduino_build
  4. Add this makefile to arduino_build:

.

#<Copyright Header>
#Copyright (c) 2012 Jordan "Earlz" Earls  <http://lastyearswishes.com
#All rights reserved.
#
#Redistribution and use in source and binary forms, with or without
#modification, are permitted provided that the following conditions
#are met:
#
#1. Redistributions of source code must retain the above copyright
#   notice, this list of conditions and the following disclaimer.
#2. Redistributions in binary form must reproduce the above copyright
#   notice, this list of conditions and the following disclaimer in the
#   documentation and/or other materials provided with the distribution.
#3. The name of the author may not be used to endorse or promote products
#   derived from this software without specific prior written permission.
#
#THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
#INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
#AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL
#THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
#EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
#PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
#OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
#WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
#OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
#ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
#</Copyright Header>

HDRS = Arduino.h binary.h Client.h HardwareSerial.h IPAddress.h new.h pins_arduino.h Platform.h Printable.h Print.h \
    Server.h Stream.h Udp.h USBAPI.h USBCore.h USBDesc.h WCharacter.h wiring_private.h WString.h


OBJS = WInterrupts.o wiring_analog.o wiring.o wiring_digital.o wiring_pulse.o wiring_shift.o CDC.o HardwareSerial.o \
    HID.o IPAddress.o main.o new.o Print.o Stream.o Tone.o USBCore.o WMath.o WString.o

#may need to adjust -mmcu if you have an older atmega168
#may also need to adjust F_CPU if your clock isn't set to 16Mhz
CFLAGS = -I./ -std=gnu99  -DF_CPU=16000000UL -Os -mmcu=atmega328p
CPPFLAGS = -I./ -DF_CPU=16000000UL -Os -mmcu=atmega328p

CC=avr-gcc
CPP=avr-g++
AR=avr-ar


default: libarduino.a

libarduino.a:   ${OBJS}
    ${AR} crs libarduino.a $(OBJS)

.c.o: ${HDRS}
    ${CC} ${CFLAGS} -c $*.c

.cpp.o: ${HDRS}
    ${CPP} ${CPPFLAGS} -c $*.cpp

clean:
    rm -f ${OBJS} core a.out errs

install: libarduino.a
    mkdir -p ${PREFIX}/lib
    mkdir -p ${PREFIX}/include
    cp *.h ${PREFIX}/include
    cp *.a ${PREFIX}/lib

And then just run

make
make install PREFIX=/usr/arduino (or whatever)

And then to make use of the compiled libraries and such you can use a simple makefile like this:

default:
    avr-g++ -L/usr/arduino/lib -I/usr/arduino/include -Wall -DF_CPU=16000000UL -Os -mmcu=atmega328p\
      -o main.elf main.c -larduino
    avr-objcopy -O ihex -R .eeprom main.elf out.hex
upload:
    avrdude -c arduino -p m328p -b 57600 -P /dev/ttyUSB0 -U flash:w:out.hex

all: default upload

Also, if you try to compile the libraries in libraries/ you'll get a linker error if you don't do things in the right order. For instance, I had to do this to use SoftwareSerial:

avr-g++ -L/usr/arduino/lib -I/usr/arduino/include -Wall -DF_CPU=16000000UL -Os -mmcu=atmega328p\
   -o main.elf main.c -lSoftwareSerial -larduino

The -larduino must be the last library on the command line

Anyway, this was a pretty easy way to compile it for me. As future versions of the Ardunio come out, this makefile should be fairly future-proof, requiring just a few modifications to OBJS and HDRS. Also, this makefile should work with both BSD make and GNU make

Also, if you're like me and you weren't interest in building the library, just wanted to download a copy of the binary, here is a copy: Arduino library (built using the "standard" pins_arduino.h)

Posted: 03/22/2012 19:24:54

Yay Comment Support Yay

I finally got around to adding comment support to this website after just a tiny bit of frustration (Thank god for Linode's backup service).

Anyway, the comment support is pretty basic. Just HTML encoded text. No links or anything, though I do eventually want to support a basic version of Markdown... Also, we'll see if my spam preventer is good enough. I kind of doubt it is, but I have a kill switch if I get flooded with spam.

So go on, test it out and see if it works!

Edit:

Also, yes, I plan on fixing the comment links on the front page to actually align. That didn't show up in my test environment. Right now It's 1:30am though and I'm going to sleep since this update took much longer than I wanted

Posted: 03/16/2012 06:22:56

AT&T U-Verse Fun

So I got a letter in the mail a while back saying my neighborhood is being migrated to AT&T U-Verse and there would be new equipment required and faster speeds...

Well, I didn't like the sound of "new equipment" so I googled what kind of equipment I might get. What I ended up with is this: AT&T Motorola NVG510. As you can see, there is no official page for this modem. No official manual, specifications, help guide, anything. The only semblance of documentation is this draft copy of the manual sent for FCC testing.

There are numerous limitations and other problems as well.

  1. You can't set which DNS servers gets handed out by dhcp (must use AT&T's buggy servers )

  2. No UPnP support. Not a problem unless you know, doing router bridging is non-trivial

  3. See above about router bridging being way more difficult than it should be

  4. Extremely frequent line drops. I had a total of one time of my internet going out with regular DSL over nearly 2 years. Since getting U-Verse, my connection drops multiple times per day.

I finally got the modem to do "bridge" mode and forward to my OpenBSD router. Yet another problem I found with the modem is it gives out the wrong subnet mask for it's public IP DHCP server. It handed out 255.255.255.255 when my fixed IP was X.X.X.80 and the gateway IP was X.X.X.1. It obviously needed to be 255.255.255.0

If cable internet was available in my town, I would have switched to it by now.

Posted: 03/14/2012 19:55:02

Yay Memes

I couldn't find a proper what I really do meme for programmers... so I made my own. Enjoy:

Programmer

Posted: 02/29/2012 00:24:22

Unity3D Failure

Despite Unity3D implying that Android deployment was available in the free version, they didn't specify that you also had to buy a $400 product to do it. So, it's back to the drawing board for that.

So, now I am looking at PhoneGap which lets me use HTML5 and Javascript for writing almost-native Android applications. I'm still researching, but it looks to be the best way to go for me. I already know a lot of HTML and Javascript, and I've been interested in experimenting with some HTML5 features. So, it appears to be the best of both worlds

Posted: 02/05/2012 00:25:18

Not Dead

Don't worry faithful readers, I've not abandoned you! I've been busy trying to find a job, and doing some freelancing in the meantime. I don't plan on getting comment support for this site anytime soon, but maybe one day!

Also, as you can tell, that OpenBSD thumbdrive project never came to fruition. My current personal projects include: Yet Another Scripting Language, and an Android game using Unity3D. I started out doing some Android programming in Java... but I really am not too fond of Java, so when I saw a way to make a game for Android writing C# and it not costing $400, I knew I had to check it out

Posted: 02/02/2012 21:58:08