Running a client/server ping test between two vps servers with 4 procs and 4GB of memory I saw ~30k reqs/sec.  Interestingly it performed the best when I did not enable more cores with runtime.GOMAXPROCS(runtime.NumCPU()). Enabling more cores on either the client, the server, or both reduced the reqs/sec by several thousand. I suspect it’s due to the test RPC method basically doing very little work. Given a heavier task per RPC method, ramping up GOMAXPROCS probably has a positive impact on performance.

Here’s how I got it working:

package main
 
import (
	"log"
	"net"
	"net/rpc"
)
 
type System int
 
func (System) Ping(message, reply *string) error {
 
	if *message == "ping" {
		*reply = "pong"
	} else {
		*reply = "wut?"
	}
 
	return nil
}
 
func main() {
 
	sys := new(System)
	server := rpc.NewServer()
	server.Register(sys)
 
	l, err := net.Listen("tcp", ":8080")
	if err != nil {
		log.Fatal("listen error:", err)
	}
 
	server.Accept(l)
 
}

package main
 
import (
	"log"
	"net"
	"net/rpc"
	"sync"
	"time"
)
 
func main() {
 
	reqs := 100000
 
	address, err := net.ResolveTCPAddr("tcp", "localhost:8080")
	if err != nil {
		log.Println(err)
		log.Fatal("Unable to resolve address")
	}
 
	conn, _ := net.DialTCP("tcp", nil, address)
	defer conn.Close()
 
	client := rpc.NewClient(conn)
	defer client.Close()
 
	w := new(sync.WaitGroup)
	w.Add(reqs)
 
	start := time.Now()
 
	for i := 0; i < reqs; i++ {
		go func() {
 
			var reply string
			err := client.Call("System.Ping", "ping", &reply)
			if err != nil {
				log.Fatal("ping error:", err)
			}
			if reply != "pong" {
				log.Println("ping error: did not receive pong")
			}
			// log.Println(reply)
			w.Done()
		}()
	}
	w.Wait()
 
	run_time := time.Since(start)
 
	log.Printf("total time: %s\n", run_time)
	log.Printf("reqs/sec: %.0f\n", float64(reqs)/float64(run_time.Seconds()))
 
}

While this is not in the Varnish module format yet, my still-evolving entry is here:  https://github.com/jasonmoo/skwurly

The challenge:

Take an input const char* url:

/test.php?b=b&c=bb&a=3&a=4

and output in a standardized format:

/test.php?a=3&a=4&b=b&c=bb

The goal here is to standardize a url so that it may be used as a key in a hash function of an http cache, thereby preventing duplicate pages in cache for the same url with differently ordered parameters. The sorted order is optional, so long as a canonical url is derived from any ordering of query string parameters.

It’s worth noting that since this was a friendly competition, much code sharing and talking over strategies and optimizations took place over the course of the week.

Initial strategy
My initial strategy was slow and inefficient. Scan the url and count the number of ampersands so that I can allocate an array of structs of sufficient size. Scan again to grab pointers to the first character in the param and the param length and store them in the structs. Using qsort and strcmp sort the array of structs in alphabetical order. Allocate a string of duplicate size to the input. Iterate over the param struct and using pointer assignment to replace each character in sorted order.

With the exception of the return string all of the memory used is stack allocated. This attempt produced a program that could sort ~400k urls/sec on my macbook pro. Not bad, but not nearly as good as it gets.

After getting a code review from a few other c devs I set out to take a different tack. The goal was only going touching each character of the input url once. I very nearly achieve that and it’s brought the performance up to ~1.8m urls/sec on my mbp.

This is the breakdown of all the optimizations that I tried and kept/discarded. Many great devs assisted me in this process. They are listed at the bottom of the page and I am very grateful for the time they spent with me!

Memory Matters

So it turns out the two arrays (one of const char*, one of unsigned short) performs better than an array of pointers to structs containing the same value types.  The two values to track are the starting character position of the param in the url and it’s length.

struct param {
    const char* start;
    unsigned short length;
}
param* params[];

vs

const char* params[];
unsigned short param_lengths[];

This was a bit of a surprise to me. I had assumed that sorting a single pointer array would be faster than keeping track of two arrays. I was wrong. I tried different ordering of the elements of the struct, using a long to track length as an attempt at better memory alignment, doing array style access to elements vs incrementing pointer access. Each had different impacts but at the end of it all using two arrays was just faster. Also faster was an unsigned short array vs an int array.

Variable Locality
Another optimization was to copy a const char* value into a char when I needed to test it multiple times. I had heard that this may be the case due to register use. I found it to be a minor increase in speed.

Testing for zero
When iterating through a loop and testing that my index was less than the number of elements that were in the loop, I thought I was being efficient. But what I found was taking the number of elements and assigning it to a variable and decrementing that variable each time through the loop and testing when it was zero was faster. It’s a cheaper test to test for 0 than to compare two numbers.

Pointer arithmetic
I have always heard of pointer arithmetic but now I understand why it’s so useful. Taking a pointer to the first element in an array and incrementing it to iterate through it’s elements is elegant and more efficient than taking an integer i and using it as an array index. I wish more languages afforded this.

Hashing costs
A great suggestion from a friend was to hash each param to an integer and sort by the integers. In theory this would allow for touching each character of the input url only once before sorting was possible. I used a cheap hash called djb. I inlined it to reduce any function call overhead.

// djb hash
int hash = 5381;
while (*url && *url != '&')
    hash = ((hash << 5) + hash) ^ *url++;

What I found was that it was a little bit slower. Why? Aside from the extra arithmetic that was being processed on every character, most query string parameters are sortable by looking at the first character of the param name only. name=jason&job=dev&company=vimeo for example, doesn’t require looking at any but the first character of each param. So I opted for character comparison.

strcmp vs hand rolled
Unforseen to me, a hand rolled string compare function performed better than native strcmp. I assumed that a native func would be all kinds of optimized but in the end a simple k&r style character by character comparison performed better than strcmp. Dumping the assembly for both showed many more instructions for the strcmp version.

qsort vs insertion sort
I set out to use as many native funcs as I could but again found that it’s not always the most optimal path. A hand rolled insertion sort performed better than the qsort native function. But I found an even more performant way to sort params that reduced constant time and allowed me to return early when a url was already in sorted order.

double-wide insertion sort
I was at a bar in the east village called double-wide and after much merriment came home to work on my url sorter. I had the thought that if I used an array of twice the size of the params, I could set the middle element as the zero index and this would allow me to prepend and append elements in O(1) time. This would save having to shuffle the entire array up one every time I had to put an element at the beginning. While this would be easily implemented in a linked list, the performance was still better with a double-wide array in my use case. I only needed to keep track of the head and tail elements of the array for easy iteration over them later. And if I only appended each param, I knew the params were already sorted. So I could return the original url.

(In this particular context the plugin architecture we were writing against allowed for returning the original string or a new malloc’d string)

Zero or one param
Returning early when only a single param or zero params were present was another more obvious optimization.

Optimizations that didn’t work:

Loop unrolling
In my feverish search for more and more obscure optimizations I stumbled upon a mind-bending gem called Duff’s device. On first glance I assumed my iphone had rendered the code wrong. After repeated attempts to understand, it slowly emerged in my mind. A loop unroller. In the author’s own words “disgusting”, and amazing.

send(to, from, count)
register short *to, *from;
register count;
{
    register n=(count+7)/8;
    switch(count%8){
        case 0: do{ *to = *from++;
        case 7: *to = *from++;
        case 6: *to = *from++;
        case 5: *to = *from++;
        case 4: *to = *from++;
        case 3: *to = *from++;
        case 2: *to = *from++;
        case 1: *to = *from++; }while(--n>0);
    }
}

I built a while-less version that optimized for 6 params or fewer where the switch cases fell through to a default of my original for loop. Initially I found a minor speed up. However it turned out to be due to an inefficiency in my loop code! Once fixed, they performed the same.

A fellow coder pointed out that loops have been pretty heavily optimized by compilers and unrolling almost never produces any benefit any more.

Single character comparison before a function call to str_compare
I had the idea that taking the first character of each param and comparing it inline and if they were the same, only then calling the str_compare function on the two params would cut many function calls and perform better.

if ((c == 0 && str_compare(params[TAIL], p) < -1) || c < -1)

vs

if (str_compare(params[TAIL], p) < -1)

What I found was that the extra logic was slowing things down. I learned a little bit about branching and how processors optimize code paths by executing as many possible branches as they can in parallel. So a simple function call (although declared as inline), beat the character comparison.

memmove vs pointer assigment
This one is probably due to the number of elements being dealt with when shuffling elements in my sorting algorithm, but there was no speedup, only slowdown, when using memmove to move elements in the array up before inserting a new one. Simple pointer assignment beat it.

Profiling and analysing

Late in the game a friend showed me how to profile my code. Using gprof, gcov, and valgrind gave me more information on how to tweak my code and how exactly it was being executed. gcov -b for branch tracking was an enlightening experience as a programmer. And in the next competition I’ll be reaching for these tools much earlier in the game.

Conclusion

While our competition may be near it’s end, I suspect we’ll never stop tweaking our code. Finding new ways to shave time and memory off our performance. It’s been such a great way to get acquainted with programming tactics in a language.  I’m thoroughly looking forward to the next.

At this point I am slightly behind the leader.
#1: 2.17m urls/sec
#2: 2.08m urls/sec (me)
#3 1.83m urls/sec

Once we have a winner and we build the code into a proper Varnish module, we will open source it on Vimeo’s github.

Awesome people
Thanks to Naren, Carlos, Derek, Paul, Won, Mashiat. Good hackin with you guys.

A little while back I had an idea to use Cellular Automata rules to generate random groupings of xy coordinates. This worked well as a way to generate a rock pattern for my canvas drawing. I even developed a little tool to help me refine the parameters to use that produced the pattern I wanted.

I wanted to expand on that approach to make something more abstract and vizzy. And animation seemed like the next step. So I built another tool that allowed me to play around with animating generated CA sets of cells from one state to the next.

It lets you embed js into the callbacks for the driving parameters and even saves your settings into the hash tag so you can bookmark settings that you like and link to them.

So I’m publishing them both here for you to play with.

I decided to use DropBox and TrueCrypt to setup a secure password file that was synchronized across all of my environments. The setup for both is dirt simple. I created a 10mb encrypted file called SAFEFILE in my Dropbox, then add a passwords.txt to it.

Then I evolved some code a friend threw my way for generating passwords. Creating a password and storing it directly in my passwords.txt file, and retrieving it is now a snap. The functions copy the password directly to the clipboard for easy pastin.

Adding my ~/.ssh, /etc/hosts, ~/.profile and a few other dotfiles to my Dropbox and symlinking them to my home directory keeps me standard across any environment I use.

Generate a random string 30 chars long for test.account:

jmooberry@local ~ : genpass 30 test.account
password added to passwords.txt as test.account
password copied to clipboard.
//  ole2iUmIGwDxtC9xVqPZiEr34ZJVwD

Grab the password for test.account:

jmooberry@local ~ : getpass test
password for (test.account) copied to clipboard.
// ole2iUmIGwDxtC9xVqPZiEr34ZJVwD

.profile helpers functions:

# password generator
# Usage: genpass 30 test.account.name
function genpass() { 
  if [ $# == 0 ]; then
      length=30
  else
      length=$1
  fi
  pass=$(< /dev/random strings | perl -pe 's/\W//g;' | head -c$length)
  echo -n $pass | pbcopy
  if [ $# == 2 ]; then
    echo -en "\n$2\t$pass" >> /Volumes/SAFEFILE/passwords.txt
    echo "password added to passwords.txt as $2"
  fi
  echo "password copied to clipboard."
}

# password grabber
# Usage: getpass test.account.name
function getpass() { 
  pass=$(grep $1 /Volumes/SAFEFILE/passwords.txt | perl -pe 's/.+?(\w+)$/$1/;')
  name=$(grep $1 /Volumes/SAFEFILE/passwords.txt | perl -pe 's/(.+?)\s+\w+$/$1/;')
  echo -n $pass | pbcopy
  echo "password for ($name) copied to clipboard."
}

# it's important.
if [ ! -d "/Volumes/SAFEFILE" ]; then
  echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
  echo "TRUECRYPT DIRECTORY NOT MOUNTED!!!"
  echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
fi

I recommend setting up public key authentication for ease of use.

# warm feet
function socks() { 
  networksetup -setsocksfirewallproxy AirPort localhost 9999
  ssh -ND 9999 -C username@domain
  networksetup -setsocksfirewallproxystate AirPort off
}

(One caveat: If you happen to close the window of your terminal without hitting <CTRL>-C first, it will not disable the socks proxy.)

The second drawing was inspired by a recent campout in Death Valley. The contrast of the blue sky, desert floor, and red coleman tent leave a strong imprint on me every time.

The end result should resemble the picture on the right. The page load time is a few seconds on my MBP and it looks quite a lot better in Chrome. The canvas size is determined at load so if you have the horses, full-screen your browser.

I say resemble because the drawing is generated using canvas and cellular automata, and derived from a random data set every time it is rendered.

One of the difficulties of drawing a picture of landscape is the seemingly random variations found in it. Not totally random because each element interacts and affects the others. The stone pattern on the drawing was the first thing I set out to solve.

Initially I was hoping to hit on some semi-random scatter pattern using PHI and a lot of trial and error. This produced a fairly uniform random scatter of stones. What I wanted to draw resembled clumps more than anything.

While scatter patterning away I happened to over hear a gentlemen at the coffee shop mentioning cellular automata and emergent systems as they explain the state of the universe. It’s kindof a hippie coffee shop but this guy is clearly in a league of his own. At first I dismissed it and then the idea grew on me that I could generate these clumps with CA. I had used CA for Conway’s game of life and a few other toy systems to experiment in the past and it was always fun.

So I generated a map of pixels that were randomly turned on. I weighted the initial data set to be very switched on (baby!). My reasoning was that I wanted grouping to occur within a few mutations and a seriously overcrowded starting set should do just that. The size of the canvas element was prohibitively large so I added a resolution factor. It causes striations in the starting set view but it doesn’t affect the CA mutations.

// set up psuedo random cell set
for (var i=0; i < total; i++) {
  if (Math.random() > .2) {
    cells[i] = 1;
  }
}

with resolution factor:

Let’s grind this set down with Conway’s game of life rules and see what comes out:

// mutate m number of times
while (m--) {
  var next = {};
  for (p in cells) { next[p] = 1; }
  
  for (var i=0; i < total; i++) {
    var above = i-w < 0 ? i-w+total : i-w,  // wrap 
      below = i+w > total ? i+w-total : i+w, // wrap
      neighbors = [above-1,above,above+1,i-1,i+1,below-1,below,below+1],
      active = 0,
      j = neighbors.length-1;

    while (j--) {
      active += (typeof cells[neighbors[j]] === 'number' && cells[neighbors[j]] === 1)|0;
    }
    // rule check
    // Any dead cell with exactly three live neighbours becomes a live cell.
    if (active === 3 && typeof cells[i] === 'undefined') {
      next[i] = 1; 
    } 
    // Any live cell with two or three live neighbours lives on to the next generation.
    // Any live cell with fewer than two live neighbours dies, as if caused by underpopulation.
    // Any live cell with more than three live neighbours dies, as if by overcrowding.
    else if (active < 2 || active > 3) {
      delete next[i];
    }
  }
  var cells = {};
  for (p in next) { cells[p] = 1; }
}

As you can see the first pass cleared out a bunch of em. The resolution factor has spaced the pixels apart but you can see the basics of game of life players emerging. By pass 3 it’s in a much more spare state but still larger groups than I want. By pass 8 the groups have moved away from each other a bit more and formed a decent pattern. Since this pattern is generated fresh from a random data set every time, we just need something that’s right most of the time.

What I like about this approach is that it gives some credence to the hippie coffee shop guy’s words. I don’t think the world runs on CA rules, but iterative, interactive rules seem to give rise to interesting “random” patterns.

The dataset produced from these mutations can now be used to render a rock pattern with a little magic for the coloring/size.

for (var i in cells) {
  c.beginPath();
  var x = (i*r)%w,
    y = (i*r)/w|0;
  c.strokeStyle = (['rgba(116,116,108,.7);','rgba(87,93,55,.9);'])[i%15===0|0];
  c.lineWidth = Math.max(30,50*(y/h));
  c.arc(x,y,3,0,360,false);
  c.stroke();
}

tl dr;

Using Cellular Automata rules to mutate random datasets can produce interesting and unique landscape patterns. This approach was super useful for me in drawing an impressionist picture of my favorite camping spot.

I wrote a simple tool to experiment with different factors for generating these patterns.

And here’s the picture that is generated from random CA patterns.

A friend was online at the time. I remembered he had a very gentle way of dealing with other programmers. He happens to be very smart and, as such, was in a position to review/teach most of the people at the company we worked at together. I asked him how to do it:

me:

hey i have a question. how do you refactor someone else’s code without being “that jerk who’s refactoring my code”? is it possible?

him:

That is very hard. When I approach that issue I always remember a quotation even though I forgot who said it: “To successfully critique something you must first learn to love it.”

Also I’m probably rewriting the quote somehow

If you approach someone about refactoring their code and show them that you’ve taken the time to understand why their code was written the way it was, or, if it’s super crappy, why they were in a compressed time schedule while writing it, or why it would help them to work with you to learn it better, etc. etc…. basically if you approach them with respect and compassion, and they get all pissy, at that point it’s their fault

But the best thing is to communicate period, often times I’ve seen people shrink away from communicating because they don’t want to confront the other person, and they end up rewriting the code without consulting with the original coder, or, worse, not dealing with the issue and leaving the bad code to rot. Confrontation, even if it pisses the person off, is always better than doing nothing or doing something around the person

All this canvas exposure and seeing how @darkgoyle does his maths got me in a mood to experiment! So I played around with implementing variations on a Fibonacci graph and many other different oddities involving timestamps and Phi. The one I liked the most was pretty simple. It is a rotation at 160 degrees with a 10px increment on the line length at each iteration. Here’s what it produces:

I think it’s rad in a simple Trony sortof way. Here’s the code to generate it:

// get the canvas
var canvas = document.getElementsByTagName('canvas')[0],
  c = canvas.getContext('2d'),
  center = { x: canvas.width/2, y: canvas.height/2 };

// setup starting state    
c.translate(center.x,center.y);

// base styles
c.lineCap = 'round';
c.lineWidth = 2;
c.strokeStyle = 'rgba(0,0,255,.7)';

var f = 1;
while (f < 500) {
  c.beginPath();
  c.moveTo(0,0);
  c.rotate(160*(Math.PI/180)); // convert degrees to radians
  c.lineTo(f,f);
  c.translate(f,f);
  c.stroke();
  f += 10;
}

A few noteworthy changes.

1. PDO now throws exceptions by default. The standard default in PDO is to fail silently. This was unintended and thanks to @jimplush for reporting it. Be sure to try/catch your code appropriately.
2. Db::execute() method added. All of the Db methods are publicly accessible and this method is for executing sql that doesn’t require a fetch (inserts, updates, deletes, etc). The requirement for this method became clear after turning on exceptions in PDO.
3. Better docs. The docs in the README have been updated and reformatted.

DumbledORM received a lot of great feedback and attention after it’s release. Thank you for all the positive response to this little ORM. I hope it continues to grow in use and remains small in size.

Download DumbledORM and get started here:
https://github.com/jasonmoo/DumbledORM

MacPorts. If you’re not already familiar with it, I recommend reading up before proceeding.

Since I already have MySQL set up and configured I won’t bother covering it here. It’s pretty well documented around the webs. The main focus here is going to be getting nginx installed and setting up PHP as a service to it with an emphasis on local development. No need to tweak this for production on my Macbook Pro.

PHP 5.3.3 introduced PHP-FPM (FastCGI Process Manager) to the codebase. Building it requires a few extra config flags and generates a php-fpm binary. The binary will manage spawning cgi processes and handling the FastCGI passthru from nginx. Currently MacPorts does not have an option to build PHP with PHP-FPM. After some hacking on the current Portfile I arrived at a working solution. It includes the correct flags, a dependency on libevent, and a startup item.

First things first.

sudo port selfupdate

# recommended but probably not a requirement
sudo port upgrade outdated
sudo port uninstall inactive

(If you’ve already got the php5 package installed, uninstall it and it’s cohorts to ensure a clean install process going forward.)

This is a bit of a hack and there’s probably a better way, like submitting a proper patch to MacPorts, but this will get the job done for today.

# replace this file with the Portfile from the gist below
/opt/local/var/macports/sources/rsync.macports.org/release/ports/lang/php5/Portfile

https://gist.github.com/724449

Then run the regular port install.

sudo port install nginx php5 +fastcgi php5-apc php5-mysql +mysqlnd

Next we’ll need to set up the configs. There’s two basic configs that require attention. First is the nginx config. Since this is just for a local development env we can be pretty lean on what we need.

# /opt/local/etc/nginx/nginx.conf

worker_processes  1;

error_log  /var/log/nginx/error.log;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile           on;
    keepalive_timeout  65;

    server {
      listen 80;
      server_name   ~^local\.(?.+?)\.com$;
      root  /Users/jason/Sites/$site/web;
      index  index.php index.html;

      location = /favicon.ico {
        log_not_found off;
      }

      location ~ \.php$ {
         fastcgi_pass   unix:/tmp/php-fpm.sock;
         fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
         include        fastcgi_params;
      }
    }
}

This is the config that I’m currently using as my generic site config. It has a few nice features. First the server name is matched against a regular expression. So any site that matches local.xxxx.com will be parsed and use the xxxx as a sub-directory in my Sites folder. This is nice. So I only need to create a new directory and add my local.xxxx.com domain to /etc/hosts pointed at 127.0.0.1 and I can immediately begin developing. Gone are the days of creating a new v-hosts file, enabling it, and restarting apache every time I want to play with a new site’s code. The other nicety here is we’re using a unix socket for the fastcgi passthru. This is unnoticeably faster on my Mac, but again frees up the need to be concerned with another open port on my system.

It’s worth noting that any fastcgi_param that you define in this config is available within PHP as a $_SERVER variable.

The second config is the FPM config. Copy /opt/local/etc/php-fpm.conf.default to /opt/local/etc/php-fpm.conf and open it up. Here are the important ones to modify:

pid = /opt/local/var/run/php-fpm.pid
error_log = /opt/local/var/log/php-fpm.log
listen = /tmp/php-fpm.sock
listen.owner = _www
listen.group = _www
pm.max_children = 1
pm.start_servers = 1
pm.min_spare_servers = 1
pm.max_spare_servers = 1
pm.max_requests = 500
slowlog = /opt/local/var/log/php-fpm.log.slow

It’s pretty straight-forward. Lean cus we can. Save and continue.

At this point we’ve got all the pieces in place. We just need a way to start/stop/restart nginx and FPM. I looked for a cleaner way to accomplish this but in the end it came down to a handful of aliases. Add these to your ~/.profile.

# nginx
alias nginx_start='sudo launchctl load /Library/LaunchDaemons/org.macports.nginx.plist' 
alias nginx_stop='sudo launchctl unload /Library/LaunchDaemons/org.macports.nginx.plist' 
alias nginx_restart='nginx_stop; nginx_start;' 

# php-fpm
alias fpm_start='sudo launchctl load /Library/LaunchDaemons/org.macports.php-fpm.plist'
alias fpm_stop='sudo launchctl unload /Library/LaunchDaemons/org.macports.php-fpm.plist'
alias fpm_restart='fpm_stop; fpm_start'

Reopen the Terminal window to enable the new aliases. Start or restart nginx and FPM as the case may be.

At this point you should be able to create a site directory, add a domain to /etc/hosts and drop in an index.php with phpinfo(); to verify that PHP is working.

Hopefully this will be of some use to other aspiring MEMP devs out there. I found it intensely fun.