Reasons to Keep a Blog and How to Get Beta Users

Number one, if you like to write and I do, then it’s fun. But as Steve Yegge outlines in his post “You Should Write Blogs” it’s also good for you. I won’t repeat Steve’s points but I do want to add one point and that is a blog is a very easy way to keep track of little tidbits of information that could otherwise be hard to refind. As an example all my posts on mono, nginx, asp.net, and asp.net mvc were written purely because I did not want have to try and find the information again or try and dredge it up from my poor memory. Instead I wrote the posts and now whenever I need to reconfigure a server to use nginx with mono and asp.net I can easily find the information on my blog. In a way it’s an extension of your memory, it’s a permanent easily searchable record of what you have done.

In that spirit I’m linking to Patrick Swieskowski and Sascha Kuzins’ post, How we got 18,000 beta users in 4 weeks. Now when I need to look up how to get beta users I can simply search my blog and find this article.

Statistically Accurate Ratings

In a previous post on ratings I noted some issues with using the mean vs the media for the rating. A few days ago Jeff Atwood posted on user ratings and specifically how to sort a set of items that are rated by users. Jeff’s post included material from Evan Miller’s article, How Not To Sort By Average Rating, on the same subject.

Below is the relevant portion of Evan Miller’s article:

CORRECT SOLUTION: Score = Lower bound of Wilson score confidence interval for a Bernoulli parameter

Say what: We need to balance the proportion of positive ratings with the uncertainty of a small number of observations. Fortunately, the math for this was worked out in 1927 by Edwin B. Wilson. What we want to ask is: Given the ratings I have, there is a 95% chance that the “real” fraction of positive ratings is at least what? Wilson gives the answer. Considering only positive and negative ratings (i.e. not a 5-star scale), the lower bound on the proportion of positive ratings is given by:

(For a lower bound use minus where it says plus/minus.) Here p is the observed fraction of positive ratings, zα/2 is the (1-α/2) quantile of the standard normal distribution, and n is the total number of ratings. The same formula implemented in Ruby:

require ’statistics2′

def ci_lower_bound(pos, n, power)
if n == 0
return 0
end
z = Statistics2.pnormaldist(1-power/2)
phat = 1.0*pos/n
(phat + z*z/(2*n) – z * Math.sqrt((phat*(1-phat)+z*z/(4*n))/n))/(1+z*z/n)
end

pos is the number of positive rating, n is the total number of ratings, and power refers to the statistical power: pick 0.10 to have a 95% chance that your lower bound is correct, 0.05 to have a 97.5% chance, etc.

Now for any item that has a bunch of positive and negative ratings, use that function to arrive at a score appropriate for sorting on, and be confident that you are using a good algorithm for doing so.

Sadly everybody has simply quoted the mathematical forumla and not even given links to the material on how to derive the formula. I was able to find an article by Keith Dunnigan here that gave an outline of how it is derived along with several other confidence intervals. Hopefully later I can take a look at my textbooks and do a full derivation.

How To Auto-login Using ASP.Net FormsAuthentication

For our demo website, demo.quickpm.net, I need to have a default automatic login. Using the ASP.Net Membership and FormsAuthentication an easy way to do auto-login is the following code

string userEmail = "john.doe@gmail.com";
string password = "thepassword";
if (Membership.ValidateUser(userEmail, password))
{
      FormsAuthentication.SetAuthCookie(userEmail, true);
      Response.Redirect("~/");
}

The above code assumes that “john.doe@gmail.com” is a user and “thepassword” is their password.

Mono Asp.Net MVC and Nginx

I’ve been using Mono ASP.Net for a while with Nginx as the server. I recently started playing around with ASP.Net MVC and discovered that it doesn’t play nice when using Nginx as the server. To make ASP.Net work with Mono and Nginx you need to download the Mono source from here. And then modify the file mcs/class/System.Web.Routing/System.Web.Routing/Route.cs
in particular comment out the following lines in the GetRouteData function

if (pathInfo != String.Empty)
throw new NotImplementedException();

If you comment out the above two lines your MVC website should work just fine when hosting it using Mono and Nginx. I don’t know if there are any negative consequences for commenting out the above two lines but I have not encountered any.

Book Memes

Book memes
* Grab the nearest book.
* Open it to page 56.
* Find the fifth sentence.
* Post the text of the sentence in your journal along with these instructions.
* Don’t dig for your favorite book, the cool book, or the intellectual one: pick the CLOSEST.

“Fibonacci numbers are related to the golden ratio phi and to its conjugate, which are given by the following formulas: phi = (1+sqrt(5))/2, conjugate phi = (1 – sqrt(5))/2″

I’m way too geeky most of my books are mathematics and computer science.

Mono FastCGI automatic subdomain apps

As I outlined in my previous post on using Mono with FastCGI and Nginx, I’m hosting multiple ASP.Net applications with Mono. I already have Nginx configured to automatically direct subdomains of quickpm.net to the fastcgi-mono-server2 program using the fastcgi protocol.

I also have the fastcgi-mono-server2 configured using the following options
fastcgi-mono-server2 /applications=cmd.quickpm.net:/:/usr/local/nginx/html/cmd.quickpm.net,quickpm.net:/:/usr/local/nginx/html/quickpm.net /socket=tcp:127.0.0.1:9000

To quickply explain, the /applications argument is setting the cmd.quickpm.net domain to be served by the directory /usr/local/nginx/html/cmd.quickpm.net, and the other sites are set to their respective directories. The /socket option is telling fastcgi-mono-server2 to listen using the tcp protocol on port 9000 of the local host.

My only problem is that adding new subdomains is not automatic. If I want to add a new subdomain to my website I have to shutdown the mono fastcgi server and restart it with the subdomain added to the /applications argument, I could use a config file instead of the command line option but the point is I have to shut the server down and restart it to add subdomains. I want avoid doing this. What I want is an option like the following /applications=%1.quickpm.net:/:/usr/local/nginx/html/%1.quickpm.net so all subdomains are automatically handled.

The only way to do what I want, I believe, is to patch the mono xsp2 source code.

Configuring Mono and ASP.Net on Nginx

For work I need to setup one domain with multiple subdomains. The basic idea for the website is the same as Fog Bugz, in that it’s a web hosted app. And each customer has their own site e.g. johnny.quickpm.net would be johhny’s site.

For the technology side we use ASP.Net Web forums for the actual software, that is johnny.quickpm.net is running an instance of Asp.net WebForms on Mono.  But for the main site quickpm.net I want to run ASP.Net MVC since I like the MVC model and mono now (mostly) supports ASP.Net MVC in the current daily builds.

For the server software I’m using Nginx and the Mono FastCGI interface.  The important configuration bits are
nginx.conf

#user  nobody;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] $request '
    #                  '"$status" $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

    server {
        listen       80;
        server_name  localhost *.quickpm.net;
	root /usr/local/nginx/html/$host;
        #charset koi8-r;

        access_log  logs/access.log;
	location / {
		 root /usr/local/nginx/html/$host/;
	 	 index index.html index.htm index.aspx default.aspx Default.aspx;
		 fastcgi_index Default.aspx;
		 fastcgi_pass 127.0.0.1:9000;
		 fastcgi_param SCRIPT_FILENAME /usr/local/nginx/html/$host$fastcgi_script_name;
	 	 include ../conf/fastcgi_params;
	 }

        #location / {
        #    root   html;
	#    index index.html index.html index.aspx default.aspx Default.aspx;
        #}

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

    }
}

To run the mono fastcgi server use

fastcgi-mono-server2 /applications=cmd.quickpm.net:/:/usr/local/nginx/html/cmd.quickpm.net,quickpm.net:/:/usr/local/nginx/html/quickpm.net /socket=tcp:127.0.0.1:9000

Doing the above results in quickpm.net resolving to /usr/local/nginx/html/quickpm.net and cmd.quickpm.net resolving to /usr/local/nginx/html/cmd.quickpm.net also mono will serve the two directories as two separate ASP.Net Applications.

Sqlite: Unable to open database file

The webapp for my project at work was receiving the error “unable to open database file” from SQLite. The error was intermittent hence it was hard to diagnose. I was using a SQLite database on a webserver running xsp2 and mono (i.e. ASP.Net).  A few times I also received the error “System.IO.File: Too many files open” but this only happened once or twice.

I was running the webapp on openSUSE 11, after some dead ends I finally tried opening a large number of file descriptors from a simple console app. That’s how I discovered that by default openSUSE 11 only lets a process open ~1023 file descriptors. But xsp2 opens a large number of file descriptors when it’s serving web pages in fact this is why SQLite was giving the error “unable to open database file”. SQLite couldn’t allocated a new file descriptor for opening the database file since xsp2 had already used all of the available file descriptors. The fix is to add the following lines to /etc/security/limits.conf

* soft nofile 2048

* hard nofile 4096

To check the user file descriptor limits on unix run the command: ulimit -a

Mt Whitney Hike

I took a long weekend to climb Mt Whitney with my Dad, the photos are at http://picasaweb.google.com/Bryan.W.Bell/MtWhitney

Music Similarity (somebody did it!)

Way back last year I did a couple posts on audio analysis in the context of music similarity. It turns out that Dominik Schnitzer of Austria has worked extensively on this project for his masters thesis. The project is Mirage