Breaking things up

The winner of the last blog entry was Django. Reading over it again, it seemed that my main objection to Django was that the problem structure - hierarchical data, stored in Mogile, with lots of functions that do things other than CRUDscreen operations (eg. compilation & asset management) - didn't really fit a web framework meant for CMSes. Other than that, Django would be great, and there were a bunch of 3rd-party Django apps I wanted to use.

But there's another option - I could make the structure of our project fit Django, by pulling all the other stuff relating to the game creation language or compiler out into a separate project. And I should probably do this anyway, because it leads to a more flexible, more decoupled architecture. For example, I could run command-line tools that make & publish games and yet don't need Pylons installed.

So I just spent a couple days pulling everything out into separate projects. I have one for the compiler (input: a JSON data structure representing the game, output: a Flash file), one for storing things in Mogile, and eventually one for the website itself. Former 2 are standalone Python projects installed via setuptools, the latter one is a Django project that hopefully will also have a setuptools script. The compiler passes all its existing unit tests for both Python and Actionscript already, the storage module will be tested in a bit, maybe after dinner.

I'm kinda surprised (and very glad) that it only took 2 days. When I did the same thing at my first employer and my last employer (hah, no matter where I work, I end up doing the same job), it took like a month, and in my last employer's case was never finished. Granted, there I needed cooperation from the other engineers, and they all had other things to do.

Pylons and Django

Thinking about switching the webapp portions of GameClay from Pylons to Django. This would be our third framework (web.py -> Pylons, Pylons -> Django), and Pylons is mostly doing acceptably, though it's pretty annoying in some places and needs some code to fill in functionality that Django provides for free. So this is not something I'm going to do lightly.

Here's a list of the various aspects and advantages & disadvantages of the frameworks:

Templates

For Pylons, I'd stick with the default of Mako. Django comes with its own templating language, but it's not terribly hard to swap it out. However, the existing contrib apps expect templates to be written in Django's language, so if we do any customization of contrib apps, we'll need to use Django's. It's probably better to stick with that for all web templates, even though we'll be using standalone Mako in the game compiler.

Advantages for Pylons:

• I don't have to rewrite our existing Mako templates
• Mako is significantly more powerful than Django, with arbitrary expressions, defs, template inheritance (beyond layouts)
• Mako filters get pulled from the template namespace automatically, which is a useful convenience when using potentially lots of filters
• Same template system for website and game compiler
• I can extend it with defs, in the template, without needing to write a separate Python module to write new tags
  

Advantages for Django:

• I like the syntax better; it seems cleaner and easier to remember
• Filters can take arguments; useful for things like indent()
• The inclusion_tag syntax provides cleaner-syntaxed and more flexible blocks of reusable content than Mako's defs. The pitfall is that these are not defined with the other templates; they live in the template_tags directory.
  

Models

For Pylons, I'd use our custom Diffle DB database wrapper. For Django, I'd use its ORM.

Advantages for Pylons:

• Simpler; I know SQL already, don't need to learn a custom model definition language
• Can handle multiple databases reasonably easily. It seems very difficult to add this to Django; there's been a branch with it since 2006, but no movement on it. It seems like it'd be a real pain to adapt save() and the query methods to a horizontally partitioned database.
• Gives finer-grained control over queries; I can often avoid some extraneous SELECT or UPDATE statements that Django's ORM adds.
• Supports multi-column primary keys, something I've found pretty useful in past DB models.
  

Advantages for Django:

• The model definition classes are actually kinda convenient to use; would eliminate the mental context switch needed to go between SQL and Python. (I can get similar benefits by moving to SQLAlchemy + Pylons, but then I lose the other advantages of using only a thin wrapper).
  
• The admin interface is very convenient; it'd let Mike respond to routine complaints from users instead of requiring my intervention. (I might be able to build something similar by introspecting the database, but it'd take time.)

Controllers
Pylons controllers are basically WSGI apps, setup by the "paster controller" script. Django controllers are functions that take a Request and return a Response. Pylons uses WSGI pretty extensively throughout; in order to understand the choices it's made for controllers, it helps to understand WSGI really well (I don't, yet).

Advantages for Pylons:

• Each Pylons controller is its own WSGI app and in theory can use any other WSGI middleware. (In practice, I'm not sure this works, since Pylons constructs the controller objects with each request.)
• Provides an easy way to attach data to controllers, like a database or mogile connection
• Routes syntax is a little cleaner and more flexible than Django's URL-based dispatching.
  

Advantages for Django:

• Don't have to use the ugly c, request, session, etc. global variables. Since WSGI apps have a defined signature, they can't take additional parameters, which can be quit inconvenient in Pylons.
• Don't need to mess with StackedObjectProxies. Globals themselves aren't suitable for use in a webapp (they aren't threadsafe), so Pylons has its own version of a threadlocal. Unfortunately this is one hell of a leaky abstraction; you can't do dir() or vars() on it and get back anything sensible, and they break when used in generators.
• More explicit. Less to keep in mind.
• Less verbose. Controllers are just functions, and they can all live in views.py (though you can break them out into separate modules if necessary). Presumably they could be callables if necessary, though that breaks the next point.
• Can be decorated. Django provides a bunch of useful decorators, like @login_required.

I much prefer Django's controller system - it's simple, intuitive, and doesn't have lots of boilerplate. Personally, I think that Pylons' approach of using globals is a big mistake; it was one of the main reasons I moved away from web.py

Sessions/Login/Registration
Pylons gives you AuthKit by default, but that looks so unpolished, undocumented, and unsuitable to GameClay that I'd probably have to write my own. Django offers the contrib.auth application, which depends on the Session middleware.

Advantages for Pylons:

• Completely flexible, since I'm writing my own. Can include things like IP addresses in the session table, or reference the sessions table to see who's playing which games, or check out what your friends are playing.
• Integrates easily with the rest of the app, eg. templates and other controllers
  

Advantages for Django:

• Provides usable default controllers for login, logout, and registration
• User objects integrated with request; no need to explicitly check for login
  
• Can retrieve all sessions through Session model; however, session models have only the key and data, and so if we want to search by user or page, we have to do it ourselves. We probably need to use custom middleware for this; we can't do it as a session backend because we don't know the username at that point, and it'd be a pain to do it as an authentication backend because we'd also need to change the Session model and that's not directly accessible.
• Handles details like tamper-proof sessions, hashing, generating unique sessions, etc. transparently
• Has convenient @login_required decorators for marking posts that require logins. Also allows filter functions; this could be useful once we have shared games and such

Other goodies

With Pylons, you're basically on your own. You can use any Python libraries you need, but you can do that with Django too. Django provides a bunch of nifty features that we might like to use:

Advantages of Django:

• CSRF detection. We need to add this to all our forms otherwise; this is pretty handy middleware.
• Redirects. Particularly if we change the ID or URL scheme. We don't want to ever break URLs; this could be a handy way to avoid that.
• Syndication feeds. There are Python libraries to do this, but the contrib app is likely easier to use

Configuration, Packaging, and Testing
Pylons is based on Paste, setuptools, and nosetests. Django has its own config and packaging system, and uses doctest and unittest for testing. The two are roughly equally easy to use - they both provide single commands to setup most things.

Advantages for Pylons:

1. They handle multiple config files very well. Each config file is a separate .ini, so you have a test.ini for testing and a development.ini for development and a production.ini for production. Each of these can be checked in, and you specify the configuration to run when you execute Paster. Django lets you have multiple settings modules, but you have to switch between them with an environment variable or command-line switch, which is kinda annoying.[edit: the command-line switch turns out to not be that annoying in practice, though a little confusing if you don't know about it.]
2. Searches for doctests in all locations, not just models and a specific test module. When you have lots of non-DB code, as we do, there will likely be lots of other stuff.
3. Easy to specify foreign dependencies via setuptools, and create command-line scripts, and all the other great stuff that setuptools does. Using setuptools with Django is somewhat clumsy; I hope it's possible.

Advantages for Django:

• Using a Python module as a config file keeps everything Pythonic, and gives some added flexibility
• The concept of separate "sites" and "apps" results in a cool architecture where we can potentially utilize other people's contributed apps. I doubt this will be terribly relevant for us, but it may turn up something neat.

Overall, I think Pylons has the advantage for packaging and distribution. I'm disappointed that Django seems to be moving away from setuptools, even dropping it for packaging the Django distribution itself. Pylons also really thought through deployment in multiple environments, while Django seems to have done a quick & dirty solution so they could go on to other things.
Other reviews

http://pythonmag.blogspot.com/2006/02/pylons-vs-turbogears.html
http://jesusphreak.infogami.com/blog/why_django
http://jesusphreak.infogami.com/blog/vrp1
http://www.cmlenz.net/blog/2006/08/the_python_web_.html

Unit testing

Finally got AsUnit to work with MTASC. There is basically zero documentation on the web for this; I should probably write something up sometime. Goes on the bottom of my TODO list, after I get everything done for Diffle.

I rigged it up so we have a Python script that builds the game archetype, then links that against AsUnit and our unit tests and runs all the unit tests through the standalone flash player. It's pretty convenient - I would've preferred a completely command-line solution that automatically checks all output and doesn't require a GUI, but this is still a single command.

It found 7 (!) bugs, just in the ActionScript skeleton we've got so far. Most were silly typoes, forgetting an element of an object hierarchy or passing in an object when we meant a string ID. But without some sort of testing, we'd never have been able to track them down. Flash is notoriously debugger-unfriendly, particularly on Linux without the Flash IDE.

Mental Centering

I'm feeling like my mental center of gravity has been shifting more towards Diffle over the past 2-3 weeks. I remember back in October I felt like I hadn't really "jumped" yet, that something was missing. It wasn't really a matter of jumping, more a gradual acclimatization as I "detoxed" from my previous day job.

I kinda want to do a tool that'd graph Subversion commits over time, just to see if this is reflected in actual productivity. It feels like it is; I think I've done something like 50 check-ins in the past week. And I'm actually moving fairly quickly on the rewrite. In the week and 2 days since the last update, I've got a blank archetype all compiling, including expression compiler and all, and the code is much cleaner than the not-incrementally-tested version. This time around, it's got full unit tests and documentation and I've figured out how to use setuptools and paster to automate the whole build process.

Some notes on the issues in the last blog posting, so I know why I made certain decisions:

1. scheme2js was rejected because:
1. It can't call out to arbitrary JavaScript functions; they have to be declared, in a way I couldn't figure out
2. It doesn't generate readable code at all - everything is temps, and it's like it's machine code.
3. I don't think there's any way I can get it working to generate MTASC code.
2. haXe was rejected because external libraries need to be declared, which is a huge extra burden when we use as much 3rd-party JavaScript as we do. Also didn't seem particularly JQuery-friendly, and the mailing list posts indicated that they had no plans to support it because it was too functional and not OO-enough.
   
3. I'm keeping the _root.fn nonsense. I still hate it, but we don't have a better alternative at this point, and we can always get rid of it via global find & replace.
4. Leaning towards invoking the compiler via AJAX for the editor issue, though I still need to see how it works in practice.
5. Canvas tag prototypes went very well - it was quite easy to use, generally cross-browser, and we got some neat effects like lasers and rotation working.
6. Didn't get around to prototyping the expression language, but I did think more about its design. There will be an actual parser for it. However, I'm going to limit to arithmetic/boolean expressions and function calls. Thinking about making it strongly-typed, with typechecking running via AJAX to tell you if you've made an error as you type. As for variable scoping, it's pretty simple: I'm introducing keywords "this" and "that", and the name of each sprite type (subscriptable) will also be a variable introduced into the current scope. Until we have user-defined variables, I don't think we need more.
7. Didn't prototype worlds larger than the stage; really should, I guess. I figure that we'll just make sprites children of a 'world' movie, and then move that around to scroll the world.
8. I got Mike's new layout up and running under Mako. It looks pretty cool, with all the drop shadows and rounded corners. Was pretty simple to write too; the HTML really is dead-simple.
   

Next steps are importing games into Mogile and building the editor. I expect some fun surprises with the editor.

Oh, and I wanted to outline the sequence of events that led up to the rewrite, in brief form, because it's instructive of the kind of design pitfalls that hit startups:

1. Working on the editor, found that I couldn't edit trajectories and have them immediately applied to the game.
2. Fix would require that the editor have knowledge of individual game archetypes, which would introduce too much complexity.
3. Dropped the concept of game archetypes with special-cased code.
4. Because of this, the base game structure needs to have many more options.
5. This changes the primary data structure we're storing games as
6. Which touches nearly every aspect of the system, from how games are stored in Mogile, to how archetypes are specified in the codebase, to the type of UI required, to the compilation mechanism.
   
7. Which means we might as well rewrite everything.

I'm generally much more satisfied with the new architecture though - it's much more flexible and elegant, and since I've eliminated the whole archetype degree of freedom, I can make the editor much more user-friendly and robust.

Rewrites

Bit off more than I could chew. It's now been a week since my last commit, which is usually a signal that it's time to do a revert and start over again. I got most of the code done and was working on debugging, but then I realized there's no way I can verify the quality sufficiently to have confidence in the code I just wrote. So it looks like I'll be scrapping it and starting with a simpler feature addition.

No matter how many times I make this mistake, I always seem to make it once more...

Actually, I may back up all the way and start from a fresh Pylons project (keeping the bottom-up widgets libraries I've developed, of course, which now account for over half the code). Now that we have a better idea of the data structures involved, many of the early attempts at a game compiler just seem like extra cruft. We can also keep the editor, compiler, and game archetypes reasonably in-sync instead of having the compiler and games get way ahead of the editor.

There are a couple of weak spots that I'm still feeling shaky about and want to prototype some more before I build something that'll hopefully be the final version:

1. Worlds larger than the stage. I haven't really played with this in Flash, yet I think it's really important for our initial version. Games are much more fun when there's more of them offscreen. We also need to figure out how to edit them in JavaScript - am thinking OpenLayers or Canvas, maybe.
   
2. Canvas tag in general. Nobody seems to use it, but it may be pretty useful for what we're doing.
3. Expression language. This is a big one; the existing system is pretty ad-hoc, consisting of simple regexp replacements on what is basically ECMAScript. We should have some sort of formal language with error checking and a real parser itself. The existing version isn't even correct in some cases.

I also need to figure out how to reflect our changes immediately in the editor, now that we offer full, composable statements. I figure our options are:

1. Create an interpreter in JavaScript for the JSON "language". Don't really like this one, because then we need to update both the compiler and the interpreter when the language changes, which seems like a pain.
2. Duplicate the compiler in JavaScript, then eval the resulting function string and place it in the appropriate event handler. Same issues
3. Call the backend compiler and have it return a result via AJAX. This seems like the best option in terms of code duplication, but will result in a delay before the change is apparent. This may be acceptable, though, as long as it's only a couple seconds.

Also, I wanted to look at scheme2js and see if there's any usage for that. I'm guessing there won't be - we want to serialize our data as JSON for its cross-language benefits, so we'd just have to compile the JSON version to Scheme. But we do need some way to write library functions that can be used with both MTASC and JavaScript, preferably without the _root.fn nonsense we've got now.

Maybe I should look into haXe too. Last time I looked, there were issues with library support - whatever we use for JavaScript needs to support JQuery and probably Canvas. But at least it'd eliminate the need for MTASC.

Platforms

X-posted from a planWorld entry:

I just followed a link to a Worse Than Failure entry, and the article was almost an exact description of my last employer. Well, not quite, since they programmed in Java and ended up reimplementing a half-assed version of Java, while the article is all SQL. But close enough.

Maybe it was a good thing that I left.

It's funny though, when I started my own company, I vowed I wouldn't make the same mistakes as my two previous full-time employers. And now I'm either making or have narrowly averted making at least a half dozen of those same mistakes. It's funny how easy it is to fall into those traps.

Note to people who are thinking of starting a software company: do not start out thinking "I'm going to build a platform for X." Because X will very quickly become "everything". And then you'll have a poor imitation of the programming language you used to build it.

At least I know a bit more about programming language design than my last boss did, and have a better idea of the tradeoffs involved. And Mike's been pretty good at reigning me in and saying "No, we're not going to support that, let's just get a basic game done and worry about extending it later." So hopefully we can find a happy medium where it's still usable to users without programming experience and yet can build cool things.

Since this is a private blog, I can get into a little more detail here. The resolution to my last blog entry was to abandon the idea of archetypes. Trajectories are just one type of stateful interaction that needs to be reset as the user edits the game, and different archetypes may have different ones (shields, for example, or other game properties). These would all differ in different archetypes, so there's no way to adapt the editor software to handle all possible cases other than to hard-code behavior for each into it.

Besides, coding up all the different archetypes would be a huge programming burden on me, and we just don't have manpower for it. Much of the code was shared between them, anyways, yet we have no facility other than our template-based shared libraries for writing routines that can be used by multiple archetypes. And it limited user freedom - if they wanted a hangman where the body parts moved and could be shot at, they were out of luck. This opened us up to user confusion, since given flexible enough customization, games could drift quite far from its original archetype. Users would be asking "Why can't we make this game do what we want, when this other one which is nearly identical can do it?"

So I ditched the idea entirely.

I haven't told Mike yet - normally we make all decisions by consensus, but in this case, the alternative is that we can't finish a real piece of software in a reasonable amount of time. I figure I'd just have to overrule him anyway on technical grounds, and I hate giving people a choice when we don't really have one.

Unfortunately, that means I've got to be doubly-vigilant about things getting too complicated and technical. So far the UI seems okay, and aspects of game design actually got simpler from this. But now I'm finding there are so many such aspects, and each has to be handled. The one that prompted the above blog entry was sprite creation - it's fairly easy technically (we've already got createAt/createOn/createIn actions that handle the various cases), but the UI offers lots of potential choices. Do we give a separate Creation tab for each sprite, where the user can specify where, when, and how many? Do we make it an action attached to a timer on the game? How do we handle randomness in creation?

JavaScript, how I loathe thee

Random complaint: state is the root of all evil. I'm trying to get all existing sprites to update their trajectories when the game property's changed, but the trajectory data is duplicated in so many places.

Of course, I knew this already (yay Haskell), but there seems to be inherent state to this problem that can't be programmed away. For example, bounce/gravity can distort a sprite's trajectory so that its present course doesn't match the specified value; this happens to a single sprite, so we can't have them all share the GameProps' value. So we're stuck trying to reset trajectories of some fraction of existing sprites.

Widgets done, integrating them

About a week ago, I was going to write an entry about the mental stresses that a startup puts on you. I was trying to figure out how to setup trajectories & actions, and create other sorts of complex behaviors where the options themselves may have parameters and recursive calls to other widgets. I felt very much like my brain was at capacity, and there was no way I could fit the problem into my head.

What a difference a week makes. We've got a fairly comprehensive widget library now, they look reasonably pretty in both IE and Firefox, and I just hooked the shooter archetype up to one.

Marc Andreesen's blog entry about the market "pulling" your startup along seems dead accurate. And it doesn't even have to be a huge market. Once you've got something up on the screen that looks like you'd use it yourself, it gets much easier to spend long amounts of time on it. It's like "Oh, look, I can fix that" and before you know it you've spent five hours on it and fixed a dozen bugs.

There's still lots to do, but I seem to be settling into a routine and getting into the project much more. It kinda reminds me of when I was little and would play with Capsella or Legos for hours at a time without realizing it was time for dinner.

Taking shape...

Something I've noticed: as the project takes shape, it starts to pull you along, so it gets progressively easier to ignore distractions. It's like the project is beckoning you to fill in the gaps. My news.YC time is way down, Reddit time is down, I've stopped obsessively checking message boards for TV shows (well, mostly - I've got the Kid Nation IMDB page up in the other tab, we'll see if I go for that or the Ubuntu VM that's also in the background after this).

Also, I wouldn't think of myself as stressed (really, I live with my parents, have plenty of cash saved up, hack all day - which I'd do anyway, and get close to 10 hours of sleep a night), but my body says otherwise. I usually judge by the number of canker sores, and with 3 plus a cough that I'm nursing, it seems my immune system is pretty shitty right now. I remember a similar phenomenon at my previous employer...the week or two when the project was done enough to appear on screen yet not finished enough to show anybody were by far the most stressful.

I wish Blogger had a "current music" field like LiveJournal. These entries would be all Mike Oldfield, all the time (well, for the past several weeks at least. Pink Floyd before then).

Archetype #2

Productivity is kinda weird. After actually being social this weekend (Homecoming + meeting up with old coworkers), I got a ton done on Monday (shooter archetype, and redid the customization aspect to factor out some duplication). Then somewhat less on Tuesday, and virtually nothing today.

Was going to write a more pensive blog entry, but I think I'll sketch out some architecture ideas instead. It feels like the software is starting to take shape, yet I don't really know what that shape is yet. We gotta put this in front of real users soon and make sure we're on the right track.

YC Rejection

YCombinator rejection #2 for Diffle and rejection #4 for me. They didn't even look at our demo, or, for that matter, any of the other hacks we'd linked.

I can't say I'm not pissed off, though really I'm pissed off as a tool for getting things done. Actually, in retrospect, I'm only halfway intelligent when seriously pissed off at the world. Like way back in middle school, or how I finished my CS major in one semester.

If I disappear from this blog for a while, it hopefully means I'm getting stuff done coding-wise. Otherwise I'm a complete and utter loser, and I wouldn't want that...

Demo done

So, as you can see (or maybe not see - it'll probably be offline by the time this blog is opened up), we got our demo done in time for the application. Finished the screencast at 6:00 PM yesterday (the app was due at 10:00 PM), then just resubmitted, ate dinner, and went to bed.

Was gonna get a lot of the minor stuff fixed today, but I found that I was feeling pretty burned out after the sprint to get the demo ready. Ended up getting stymied trying to figure out what primary key I should use for games. Hopefully my coding mojo will return.

I also have straighten out some dental insurance bit with my previous employer (ugh, previous employers), sign up for health insurance, and write Mike's two recommendation letters for B-school. The latter is kinda weird, because I'm acting against my own interests - I obviously don't want Mike to go to B-school, because I want Diffle to succeed. But I'll do it anyways, and do a good job on it, because Mike's a good guy.

YC Demo

Testing our demo:





Modify this game.

22 hours till demo time

The YC app is due in 23 hours. We've submitted a version already, but we don't have a demo on it. We've got 23 - no, a little more than 22 - hours to finish one.

We're in a much better place code-wise than a week ago. All archetypes load into the system, you can view them on the web, you can edit them on the web, and we've got cool widgets for repositioning and changing images. Working on saving and publishing now; hopefully will have that done by the time I get to bed.

The part I'm worried about is installation. We have a shitload of dependencies, and some of them (I'm looking at you, libxml2 and libxslt) don't seem to want to install. I hadn't realized how much I take apt-get for granted until I moved from my Ubuntu VM to the RedHat server.

Mike has already drawn and quartered our demo pieces. They look amazing...maybe we can actually pull this off.

So with any luck, things will work perfectly tomorrow. We'll get the missing software parts done tonight, installation fixed tomorrow, Camtasia downloaded and a demo recorded tomorrow afternoon, and I'll be all set to show Mike and resubmit our application by dinnertime. If everything goes according to plan. Hah.

Slow Going

It's been slow going for the past few days. One of those stretches where I get to work, think "Oh, I should check news.YC", look at it but see nothing new interesting, head over to Reddit, nothing there, check FaceBook and LiveJournal and Planworld and Yahoo!Finance, come back to project, get a couple more lines in, go read a book, maybe a few function definitions, out for a run, come back, write a few more lines of code, update Diffle blog, and so on.

I don't really have any working code that I didn't have on Saturday. Plenty of non-working code, though. I'm trying to pull together all the Pylons, MogileFS, MTASC, JSON, JavaScript prototypes that I wrote into a single architecture that'll serve as the base for the production system. This is hard because:

• It's a lot to keep in mind at once - storage within DB/distributed filesystem, file formats, duplication of information, how it'll be retrieved and displayed, modification, versioning, compilation, etc.
• My natural perfectionist tendencies get in the way and I freak out about this being the real thing and not some throwaway prototype I can do a half-assed job at.

In Founders at Work, one of the questions that she asked every founder was "Did you ever want to quit?" I don't really want to quit - there's really nothing else for me, because I don't think I could be happy at a regular desk job where I take orders and code up somebody else's product. But this is one of those times when I worry that no matter how hard I try, I'll still fall short.

I also worry that I haven't really "jumped" yet. A friend of mine put it very eloquently (he was talking about parenthood, having married into a ready-made family, but it applies just as much to startups):

"But that's okay. Because once the step has been made -- and J____ never made it -- it's better to be there. Making the leap ahead, taking the plunge, there's a dozen different words for it. J____ stood on the brink for a while, but decided he didn't want to jump. I jumped."

There's a sort of emotional commitment to startups over and beyond working 12 hour days and quitting the day job. I'm not sure I have it yet. For that matter, I suspect that many of the places I've worked didn't have it, given how long it took them to develop software. It's a matter of becoming one with the problem domain and cranking out code as fast as you can type, because the whole program is there in your head.

False Starts

Got a FaceBook message from Bryan asking about incorporation tips and my "first full week away from work". Has it really been that short? Actually, I think it's been two full weeks (9/11 - 9/29) and his math is just wrong. But I certainly remember the feeling of time passing really quickly at work - I remember thinking on several occasions "Geez, it's been a month? I thought it was only like a week".

Since last Monday, I've gotten MogileFS working, experimented with the Python API, installed Pylons, and written about 3-4 simple Pylons apps. Have a decent handle on Routes, Pylons, configuration, packaging now. I hope.

I guess it looks like we'll be redoing the web end now after all; basically, I want a simple blank canvas upon which to build our demos. Seemed easier than trying to integrate it in with the existing Diffle.com site.

I was thinking recently about all the false starts we've had and things we've had to redo, either partially or totally:

• I started with web.py's DB module but ended up ditching it for one I wrote myself
• I started with web.py's Cheetah integration, ditched it for standalone Cheetah templates, and then ditched them for Mako
• I spent significant amounts of time learning about internationalization and HTTP testing for Python, neither of which we really used
• We did RejectedByYC.com, which we repositioned as Bootstrapacitor, which never went anywhere.
• We launched as a regular Flash games site which never got significant adoption. The logic behind that was that we'd need those features anyway, but I'm not sure now that we do, and I think we're better off launching with only our core value proposition (the game creation engine) and adding back social features as we need them.
• Initially we were going to use Ming for the Flash compiler; Mike found MTASC and Swfmill and we went with those instead.
• I almost completely rewrote Mike's initial MTASC prototypes to fit with our general architecture.
  
• We started using Prototype.js for our JavaScript library, which was replaced by YUI, which was replaced by Mootools, which was replaced by JQuery.

I know that in every successful startup, there's a large amount of experimentation and failure that goes on before you eventually find your footing. But I worry that we may be doing too much experimentation and failure, and if some of these failures were really necessary. I mean, our reasoning sounded logical at the time, but in hindsight some of these (like RejectedByYC/Bootstrapacitor/FlashGames!Diffle) were really kinda dumb.

Eventually I'll just have to dive in, fully, and whip something up. And I think we're close to the point I can do that, but not there yet. It's like I'm still pussyfooting around the edge of the product and not quite ready to make it a reality yet.

Catalog of Mistakes

8 hours later, I've got both JavaScript and Flash versions compiling from a single source, and am working on installing MogileFS for the full architecture. Never did get to the YC app, though.

Time for a quick catalog of mistakes we've made since we started. I'll add to these as we make more, and then perhaps publish the whole thing once we've overcome them all and actually have an audience. [edit: didn't actually add more, that's what the postmortem is for, but I've added little edits with commentary]

Personnel:

• Founder on a student visa. Xin had to leave because his visa wouldn't let him work on a startup.
• Too many uncommitted founders. We started with 5 founders, with a wide degree of commitment between them. It doesn't work - communication overhead eats up any added manpower they provide.
• Quitting the day job too late. Now that I know the difference in productivity, I definitely would not have stuck with the day job this long. Then again, my opinion may change if we fail or come close to running out of money. [edit: nope, still feel the same way.]

Technology:

• web.py. It's just too poorly-supported and unpolished at this point, with some design decisions that are just dumb. Will replace with Pylons, probably, but it's a low priority. [edit: and then Django]
• Cheetah. Mako is better. [edit: and Django is...I dunno about better, but I like the Django + Django package more than Pylons + Mako]
• Prototype/YUI/Mootools. Ditched all of them in favor of JQuery.
• Writing too much from scratch. If I had to do it over again, I'd probably use Pylons or Django. Then again, writing from scratch probably gave me new perspectives on how to use the framework features, so it may be unavoidable.

Strategy/Positioning:

• Too many side projects. A couple times, I suggested doing a side project "on the off chance" it succeeds. Problem is, if the passion isn't there for the idea, you won't be able to follow-up enough to see whether it'll really succeed. We'd be better off pushing through on our main idea and doing our best to make it succeed. If it fails, *then* is the time for side projects. [edit: OTOH, many of the other opportunities that I'm pursuing came from side projects I did instead of GameClay.]
• RejectedByYC.com. Yeah, we found out that putting "rejected by" in the name of anything isn't a good way to get users. And the replacement "Bootstrapacitor" was too vague.
  
• Diffle the Flash games site - if there are hundreds of competitors already, nobody is going to pay attention to the hundred-and-1st.
• Not explaining what the site is about - it's not always obvious to other people.

Legal:

• The attempt to get an IP release and protect us from claims that we developed software while employed by someone else failed. Employer wouldn't sign it. It may be moot anyway, since it looks like we'll end up rewriting nearly everything by the time we actually re-launch.

Will add to these as more come up. Time for the Heroes season premiere now...

A week and a half of freedom

It's now been a week and a half, nearly two weeks, since I quit the day job. And so far, it's appeared that that was exactly the right thing to do.

It's not really the time issue. I estimate that I work maybe 8-9ish hours a day of actual productive time on Diffle, broken into 3 hours or so when I first wake up, another 2 hours or so before dinner, and then 3-4 hours at night. I certainly couldn't do that with a day job, but it's only a factor of two or so more than I could manage in my spare time, and my productivity is way more than a factor of two higher.

Rather, it's what it does to my attention span. Before, those 4 hours a day were broken into 45 minutes or so before work, 45 minutes before dinner, and maybe two hours after dinner when I'm all tired out. By the time I actually get into the work, the amount of usable time is really more like zero. So essentially all useful work was done on weekends.

Also, I find that I'm thinking much more clearly. I'd stumbled around for about 2 months as to an overall architecture for game editor + JavaScript version + Flash version. It came to me about 2 days after quitting my day job. My code is cleaner and nicely factored. Mike remarked at how simple and intuitive the hangman prototype I put together is.

Anyway, a quick list of things I've done since quitting:

• jQuery documentation & dependency tool
  
• YC News in polar coordinates - the cloudmaps stuff didn't really work right (despite spending a few days on it), so I replaced it with a simple polar plot during DevHouseBoston. Was generally a success; did most of it during the hackathon, cleaned it up and added a couple other features in the two days afterwards, and posted it on news.YC. Will include in our YC app as a cool hack.
• Hangman prototype - it took about 2-3 days to get up-to-speed with MTASC, Swfmill, and ActionScript. I've got a basic fluency now, along with a prototype. Converting it to JavaScript took only a couple hours, and now I'm working on a compiler to generate both from the same source.

Also thought out a distributed architecture for storing games and such.

Am about to start working on the YC app. Mike did a draft last week, now we have to condense things down, since it's too wordy.

Also should start a post with a list of all the mistakes we've made. There's a tendency to white-wash the past, so might as well get them all down while they're fresh and painful. Might be interesting, and hopefully we can avoid making them again.

Last day

Has it really been 5 weeks since last update? A quick catalog of what's happened since then.

It's so weird to look back on the last post and see "YUI toolkit". We ditched YUI for Mootools, mostly because the Windoo extensions had a resize handle implementation that we could use off-the-shelf. And then we ditched Mootools for jQuery because apparently the Mootools community is a bunch of arrogant pricks (heard secondhand, but heard from many people) and jQuery is where the momentum is now. Of course, jQuery doesn't have a resize handle plugin (or many of the things in Mootools or YUI really, though it's getting better fast), so I had to write one myself. That took like a week.

We've now got a pretty good library of jQuery stuff - 3 plugins and a documenter/dependency analyzer, which I hope to release as open-source.

I sometimes worry if we're wasting too much time on ancillary tools and not spending enough on the core product. Really, a majority of the time has been spent on tools, server setup, prototypes, learning other tools, etc. And this has been convenient in some regards (one-click deployment, w00t), but I can't help wondering what the state of the product would be if the time had been invested in it instead. And now I'm doing a little widget for DevHouseBoston3 that isn't really related to the product but which might give us a better chance of being accepted to yCombinator. I feel like I really just need to devote a big solid block of time to just pushing stuff out and get it done.

Speaking of which, today was my last day at my day job. It's kinda bittersweet, I guess - these important life transitions always are. But I really can't do both that and the startup, and I'm not willing to let the startup go. Now I'm free to work crazy hours and hack crazy stuff. Liberating, but a little scary.

The IP release was a big bust. My employer wouldn't sign it, so the approx. $1K it cost and 5 weeks it took is basically down the drain. Well, technically the cost isn't, since my pay for those 5 weeks is more than it cost. But the time hurts. I hope that it doesn't cost us our market opportunity.

Moreover, it made my boss think that we had something nefarious going on. (How could he not have heard of similar agreements? Steve Wozniak had one!) I guess no good deed goes unpunished.

Anyway, it's all behind us. Tomorrow - or maybe even tonight, depending on if I can get the cloudmaps for DevHouseBoston done tonight - I get to really put in some time on the game creation engine. It's exciting.

Incorporation etc.

Wow, I seem pretty full of myself in the last post. I stand by the general sentiment, though if I were to rewrite it, I would probably use fewer literary metaphors. And I think that at this time I basically have made my decision - I'm leaving as soon as the lawyer comes through with incorporation and IP release.

Speaking of lawyers, we met with Sam Mawn-Mahlau of Edwards Angells Palmer & Dodge on Monday. He's the same guy who litigated the Ars Digita custody battle for Phil Greenspun. It was basically a massive infodump that left my head hurting for hours afterwards - I had tons of questions, and he brought up tons of issues we hadn't even thought of. For anyone thinking of starting a company, I'd recommend meeting with a good lawyer (with experience in the particular area of business you're in) as soon as you're sure it's something you really want to do. It can be really expensive (we're expecting a bill of about $4K for this incorporation and releases) but it's generally more expensive not to take care of it.

I've been prototyping the game-creation engine this whole time. As I think I've mentioned in previous posts, we have the conversion-to-flash stuff all prototyped-out by Mike, and I got it to build on Linux. We also have a pretty decent translation to JavaScript, so you can pretty much play it using DHTML only. The next step is to get the actual customization going, which has presented some pretty thorny interaction-design issues. We want it to update the game behavior & appearance in real-time, which a.) means lots of observers and b.) means we need to specify and customize a lot of properties that don't have obvious visual representations. Without getting into scripting languages, how do you specify "When the user presses space, create a new PlayerBullet at the location of Player", for example, or let them add different bullet types?

Now I'm looking at the YUI widget set and trying to do a few prototypes with it. I want to avoid having to write all my own widgets from scratch, as there are a lot of little gotchas with JavaScript that can cause little bugs.

To really live, you must be willing to die

"You are the true master of death, because the true master does not seek to run away from Death. He accepts that he must die, and understands that there are far, far worse things in the living world than dying." -- Albus Dumbledore, Harry Potter and the Deathly Hallows

A comment by staunch on news.YC really clarified this decision for me. Startups are much like the archetypical Hero's Journey: there's the call to adventure, when you get the initial idea for it, there are sidekicks & partners, there are hopefully hands-off mentor figures, there are tests and tribulations and temptations, and if you're lucky there's a happy ending where the hero embraces his destiny, overcomes his competitors, and sells out for millions of dollars. Of course, this is real life and not a fantasy novel, so there's about an 80-90% chance that the hero goes broke and gets another day job, wiser and more skilled yet somewhat embittered.

The critical success factor, though, is that the hero first acknowledges all his doubts and fears and then embraces them. So yeah, failure is a very real possibility. In the storybooks, that means death. In startups, it merely means that nobody likes what you're doing and you're wasting your time & money, and you may go broke.

Big problems are scary. I've found that this has been a significant problem for me. I could do RejectedByYC/Bootstrapacitor in 3 weeks, because I just looked at it as a distraction from Diffle, so it wasn't important, so I could just throw things up on the screen and make it work. I could do Diffle, because once we had finished Bootstrapacitor, it was just a distraction from the game creation engine, so it wasn't important and I could get it up on the screen. I've slowed down markedly on the game creation engine, because this is what we've been wanting to do all along. What if nobody likes it? What if it flops?

I guess the answer is: so what? If it flops, I've built up skills in web design, Python, JavaScript, lots of cool AJAX stuff, etc. I can try other ideas. I have money in the bank. It's not gonna kill me to spend a year working on random startup ideas.

This also answers the question Todd asked me: when are you gonna quit? And the answer is: when we run out of ideas. As long as we have something to do, why not do it?

Objections

I had a long conversation with Frank, and then Sheldon, at work today. Dunno if it was intended to get me to stay or if it was just to let me know some of the pitfalls of startups - it seemed like a little bit of both. Anyway, I want to write down some of the objections they had so I can come back to them and we can perhaps address them.

From Frank:

1. Advertising/sales expenses. It takes money to let people know about your product. You can have the best product out there, but if folks never hear about it, they won't use it.
2. Development costs - it sucks to spend $90M and then find there's no market out there.
3. The company's business is really picking up - I could be missing out on a very exciting time.
4. We should at least validate the market by sending links to an impartial potential customer (he offered his son) before plunging in
5. Much of our hurry may be premature - we can watch our competition, see if they take off, and try to catch up with them afterwards

From Sheldon:

1. We should at least be able to convince angel investors to invest money, even if we don't take their money. If we can't convince them, how can we convince customers?
2. I'm too young with too little technical experience for this
3. It'd suck to spend all this time building a product and then find that we were building the wrong product. All that development effort would go to waste.

One of the biggest mistakes entrepreneurs can make is confirmation bias: disregarding information that doesn't fit the course of action they've decided upon. I'd really like to avoid that trap here, so I'm writing down these objections here so I can come back to them impartially. Unfortunately, almost all of these are just opinions: they aren't factually verifiable, which makes them a bit hard to test.

There's also a problem in that knowing more and knowing way less than a person both seem the same: in both cases, they look stupid to you. I mentioned this problem to my friend Doug when I first joined my last employer: there were business decisions Sheldon made that seemed a little off, but I couldn't tell whether he was dumber than me or way smarter than me. Doug reassured me that he was probably not way smarter than me, but that's not really enough to go on.

I think I have a solution to that: I've e-mailed Sheldon asking for the most influential books/websites/blogs/resources in his company's growth. Because while you often can't tell if a person's opinions are wrong (you have no idea what experience they base it on, after all), you can tell if their reasoning is wrong. If you know what they know and then can point to specific flaws in their assumptions, you can answer the question of whether you're smarter or way dumber than someone.

I'm not willing to wait and see how the competitors do. The problem with these highly viral consumer web services is that they'll sit with no adoption until you solve the whole problem, and then everyone will try to use them at once. Once you've solved the whole problem, there's no way for a competitor to squeak in until another problem manifests itself, because users are happy as-is. Most cases where people say "Oh, there was no market" really are cases where there is a market, but there do not exist services on the market that solve the whole problem. For example, YouTube could not exist without FaceBook/MySpace, easy video-editing software, and cell-phone video, because there was no easy way for users to make and edit videos and no audience for them once they did. Reddit could not exist until the rise of blogs and news sites, because there was not enough freshly-updated content to make it interesting.

Leaving

I gave notice yesterday. Went reasonably well - boss said he'd like to keep me, but he's obviously not going to twist my arm, and we left open the possibility of some consulting or part-time work in the future. I did have to listen to a long lecture about how it's too early, my chance of success in negligible, I don't have the technical chops, etc. Ah well - I did hear some things that I'll want to keep in mind anyways.

A quick selection of quotes I've heard from various people around me when I mentioned I wanted to go full-time on the startup:

"So, mom and I were talking about your startup in the car, and while we think it's great that you're trying - we honestly don't think you'll succeed." - my sister

"Don't quit your day job!" - my dad

"But how are you going to eat?" - Cindelius, a netfriend

"So you're basically going to play the lottery?" - Todd, a coworker

"If you can't even convince me that a market exists for this product, how are you going to convince investors or customers?" - Sheldon, my boss

These quotes are perhaps a bit unfair and out of context - Cin and Todd have actually been quite supportive, and even the boss & family members just want to make sure I'm aware of the risks. But I thought I'd put them up there for posterity's sake. [Edit: Yeah, and they were exactly right. ;-)]

Quitting the Day Job

This is really a work-up-the-courage-to-give-my-boss-notice post, but I also want to jot down some of my thinking, just to clarify things in my own head and make sure that I'm not making a huge mistake.

I had initially wanted to start a startup straight out of college. My idea at the time was for collaborative editing, a la SubEthaEdit but over the web. As it turns out, Google Docs has implemented the same feature now, so it wasn't a horrible idea, but it wouldn't've worked out anyway because I would've just ended up competing against Google. The reasons I didn't start it and ended up taking a day job instead:

1. Upon looking at SubEthaEdit's market again, they weren't really getting widespread adoption. There was some niche interest, but few people using it for actual work, and those who were tended not to care about the collaborative editing features. What failed for SubEthaEdit was unlikely to work for us.
2. I didn't have money. Coming out of college, I had roughly $10K saved from my year at Traxit, sitting in a tax-free municipal bond fund. I had about $3-5K in my bank account from summer work. I figured that legal fees would easily eat $5K, hosting would eat a couple hundred a month, health insurance would eat a couple hundred a month, and I really didn't want to be mooching off my parents, though I did end up living at home anyway. That didn't leave much runway.
3. I felt that I frankly lacked the experience to start a startup. Yeah, I had taken FictionAlley.org's new system from conception to completion (or actually, I only completed it in November after finishing college). But I had no idea what people would actually pay for, how legal and financing stuff works, common mistakes that startups make, how to manage programmers if we ever were to grow beyond just me, or anything like that.
   
4. I'd only taken two projects successfully to completion (Scrutiny and FictionAlley), and FictionAlley felt like it was a big stretch. I worried that I lacked the discipline to carry something as big as a startup all the way through.

So I decided to take a job at someone else's startup and learn how it's done while saving up money. Best of all worlds, except I had to leave my startup dreams behind for a couple years. With luck, I'd do some side projects and something would take off, then I could quit once it looked like it'd be successful.

Fast forward a year and a half to now. The status of each of the 4 points above is:

1. I mentioned our competitors in my last post. Sploder has a forum with a bunch of people that are pissed off because its founder is no longer maintaining the site. GameBrix has gotten some tech press coverage. MyGame has a bunch of games actually created by casual browsers. It looks like our competitors are about to get traction - if we delay any longer, they'll leave us in the dust.
2. I have close to $40K in the bank, another $20K in the brokerage account, and $35K of Akamai stock I could sell. If I were to move out and live like a grad student, the cash in the bank would cover close to 2 years, and then the stock accounts nearly 3 years more (assuming no market crash, and we'd be fucked anyway in that case). That's after paying lawyers for incorporation and buying out the silent partners.
3. From what I've seen at my job and what I've read from other successful founders, startups are simpler than I thought. It basically comes down to three rules: 1.) Make something people want, 2.) Charge more for it than it costs you, and 3.) Fix problems now rather than later. We do 3 and hopefully 1 already; 2 will have to wait because of the market, but that just means we have to keep costs very low to start out. Of course, there's a lot of complexity behind each of the individual points, but that complexity depends heavily on the particular market opportunity. It's not something you can take with you from job to job. We'll just have to develop that expertise as we go along, which means we need time and focus to work on it.
   
4. Since college, I've added Write Yourself a Scheme in 48 Hours, a Netbeans Plugin for work, Bootstrapacitor, the Execution Monitor for work, and now the Diffle flash games site to that. Each of those had places where I wanted to forget about it and read Reddit at the time. In each case, I came back and finished it.
   

I've also tried to eliminate as much risk as I could from the decision to go full-time on Diffle. The way I see it, the various forms of risk inherent in a new business are:

1. Enthusiasm risk. You may get started and then find that you don't like your business idea as much as you thought you did, once you have to start doing actual hard work.
2. Team risk. Your cofounders may have divergent goals, and the team may fall apart.
3. Technological risk. You may not be able to build what you're hoping to build.
4. Market risk. It's possible that nobody will want what you build
5. Economic risk. You may not be able to build what people want at a price they're willing to pay for it.
6. Competitive risk. Someone else may build what people want better than you.
7. Scaling risk. If you build everything way better than everyone else builds it, you may not be able to scale your infrastructure to the demand this creates.
   

Enthusiasm risk was a big worry when I started, and the reason I didn't quit the day job immediately. But I think it's fairly likely that that's behind us: I'm finding I'm more excited by the actual process of working on this startup the farther we get.

Team risk actually happened to us, but we've basically dealt with it. Our 3 former cofounders seem willing to be bought out. It was an expensive mistake, but much less expensive that allocating 50% of the equity to 3 people who aren't really interested in being day-to-day involved with the business.

We've tried to reduce technological risk as much as possible. We are live with the actual website now - it's unlikely that any surprises will crop up there. We have prototyped command-line compilation from XML to Flash, so we know we can do that, and it's nothing more than string manipulation in Python. We've prototyped building a game in JavaScript, and succeeded with adequate performance. I'm working on customizing that game in real-time through JavaScript, but it seems like that should work without major hitches.

We still have to live with market risk. Ideally, I'd stick with the day job until Diffle took off and it was clear that people wanted our system, so we'd only have to worry about scaling risk. However, in the presence of competitors, they are more likely to take off and get users if the market actually exists. By trying to ameliorate market risk, we open ourselves up to competitive risk.

Economic risk is typically not a problem for software startups, because your variable costs are so low. If people like what you build, there will be some way to monetize that profitably. Often it involves selling out completely to a big company, but software that gets used almost always is a net economic positive.

Finally, we're still very exposed to scaling risk. But that'll only get worse if I still have my day job. And if we get to the point where this is a problem, I'll be jumping for joy.

I figure that worst case, I try this for a year, see how far we get, and if it was a complete mistake, I go get another job.

Competition

We've found 4 competitors in the past week and a half: MyGame, GameBrix, Sploder, and GameScene. Each of them is more advanced than us. In GameBrix's case, they've been working on it since at least January 2006, and probably have some sort of funding.

Still no more traction on Diffle.com. I can't really blame people: right now, it has nothing to recommend it over any other site on the Internet. There're a bunch of features I'd like to implement, but it's hard to get a good continuous block of time. I typically have about 20 minutes in the morning while eating breakfast, a half hour before dinner at night, then in theory I've got 2-3 hours before going to bed, but it's usually broken up into hour chunks by shower/dishes/taking out the garbage/etc. It's hard to get into flow with only an hour, particularly at the end of the night.

(For a while I was doing okay by timeshifting things and going to bed immediately after dinner, then working in the morning. But this only worked while things were relatively light at work. I find that I need more sleep the more that I program, so I was going to bed at 10 and waking up at 8, the same time I would've woken up had I gone to bed at midnight.)

I've decided that I'm going to quit my day job. Planning to give my boss advance notice tomorrow, though it may be a month or two until I'm actually done. We have to get an IP release signed before I leave; Mike is talking to a lawyer tomorrow night.

We're going to try for yCombinator funding again. Obviously I hope we get it, though having been rejected by them 3 times before, I'm not too optimistic. Mike will quit his day job for the program if we get funding, in February if not.

Still no more traction on Diffle.com. I can't really blame people: right now, it has nothing to recommend it over any other site on the Internet. There're a bunch of features
I want to implement, but I don't have a solid block of time to implement them.

Launch

Long stretches of time without posts are good things. It means I'm working on coding rather than blogging.

I suppose I should've put up an entry about the launch when we launched, but a.) I was too busy and b.) there really wasn't a clear launch date...we sort of slid into it, and we're still sorta sliding into it. This just happens to be the first time I've got time to blog instead of code. So I'll try to reconstruct what's happened since the last entry, almost a month and a half ago. There's bound to be a useful lesson in here somewhere...

We went live on Tuesday, June 26. "Went live" as in "Okay, password protection is off, you can tell your friends about it." I suppose our official launch date was July 1st: that was when Mike put one of our games on Digg to test the waters.

As of the last post (May 31), we were "Just about to launch - this is the last feature we need, really." This seems to be a common feature of startups: FictionAlley was "ready to launch in a couple of days" for about a month before it actually launched, my day job has been "ready to launch" for the last two months but with little actual activity on that front, and Diffle was "one day away from launch" for about 3 weeks. So it's worth looking at the Subversion commit log to see where that time was actually spent.

On May 31, I was working on auto deployment. I didn't actually commit those changes until Jun 2, my birthday, and then it's only "beginnings of auto-deployment". I finished (with a big ? in my commit log message) on June 5, then we had to fix bugs in the main production script that only showed up after deployment. June 7 involved adding the ability to change passwords (oops! how could I forget that), June 8 was some bugfixes, and June 9 began a pretty page for handling internal server errors, which I didn't actually finish and debug until June 12 (there were no commits between June 9 and 11 - I was at Chris & Jess's wedding).

The next week was all bugfixes - there were a lot of them, apparently. I think I actually removed the password protection on June 20 (the same day I added the custom 404 not found page - there are 8 commits that day, I must've did a whole lot of other bugfixes too). There are a couple fixes there that IIRC came from suggestions by Mike's girlfriend Sandy, so it would've been open to at least her. Then I had to convert all templates to precompiled Cheetah templates to fix a unicode encoding issue - this was the last bug before launch, I discovered it about an hour before we were about to go live and decided it was enough of a showstopper to postpone the launch. It took about 2 days to convert everything to precompiled templates, then we went live on June 26, though we didn't actually tell anyone about us.

I went on vacation - no Internet access - for the 4th of July week, which was one of the main reasons we didn't actually tell anyone. Spent the time prototyping the game creation engine - I should jot down some thoughts on that soon, because they're undoubtably wrong. Mike dugg a game anyway, to see the response. We ended up getting like 700 unique visitors from that, but the site really isn't sticky. The vast majority (like 93%) came and left without checking anything else out. Since then, we've been holding steady with about 20-40 uniques a day. I suppose that's somewhat encouraging, that at least *somebody* checks us out daily, though it's not much. Mike and I both think we really need the game creation engine in order to take off.

Aside from that - we had to switch from Cheetah to Mako because Cheetah offers very poor handling of AJAX fragments. There's no real facility in Cheetah for libraries of mixin HTML fragments: basically, each def needs to be either on the page or in a parent class. Mako is much more flexible, with both inheritance and namespaces, and also seems to have more robust unicode support and an easier API.

I should start the main Diffle announcement blog so it's not just a blank page. It's at diffle.blogspot.com, and will be the public face for announcements and status updates and such. This will continue to be more internal reflections and so on.

Doing nothing, really

I spent most of last weekend watching basically all of Heroes season 1. I did get some stuff for Diffle done (like a utility to do diffs of database schemas, and another one for diffs of directory trees), but far less than I would've liked to. My brain basically felt fried all through the long weekend.

I hear stories about "the startup life" and how people basically work 24/7, and wonder if this means Diffle is fated to mediocrity. I guess nobody will be reading this if that's the case, so I shouldn't worry too much about it. I've found that I can put in couple-week-long stretches of fairly hard work (moreso because the darn day job means I'm fighting fires all day at work), but I crash afterwards and just basically veg out for a week or so.

Still, I guess we've gotten quite a bit done since the last entry, nearly a month ago. We put about a week of work into Bootstrapacitor after the second launch before giving up. I suppose I should write a post-mortem for that. Then I did the layout for Diffle and got far more intimately familiar with HTML/CSS than I ever wanted. Mike was working on the shooter game for the full Gameclay site during this time...it looks pretty impressive already. Then there was performance testing, some indexing, some missing features like prefs and the feedback form, and auto-deploy. Am working on auto-deployment now; it's the last major thing we need in place before launch. Well, that and figuring out what's wrong with our e-mail server.

Bootstrapacitor

Relaunched Bootstrapacitor yesterday evening. Response is still a resounding thud.

There's some feedback, but still no users. I suspect the feedback is masking the real problem: people just don't need it. The story doesn't fit into people's lives. So even if it looks like a good idea, they won't use it.

Gonna switch focus to Diffle for a bit. A lot of the code we wrote for Bootstrapacitor actually consists of bugfixes for the Diffle framework & libraries.

RejectedByYC.com

I launched RejectedByYC.com yesterday at 3:00 AM, to a resounding thud. Response was generally negative, mostly because people didn't like the RejectedByYC name and logo. Too negative, too close to yCombinator, and basically too much of a ripoff.

So Mike and I brainstormed some ideas. Eventually we came up with Bootstrapacitor, from "Bootstrap" and "Capacitor", the idea being that we store up energy until the startup explodes in a bolt of lightning. There're also neat potential puns with the flux capacitor, which after all is shaped like a Y(Combinator). And we're not going back to the future, we're inventing the future.

Yesterday I added the stats page, and this morning I moved the announce link to the layout (for those who've already registered startups) and added Markdown support. Mike came up with a logo last night, so I may try to add that in today and adjust the layout. And we've already registered bootstrapacitor.com, so the site needs to be switched over to that domain.

We are facing an issue over time allocation, though - in many ways, Bootstrapacitor is a distraction from our main business, which is Flash game creation. Mike has made a lot of progress on the game creation engine, so we'll want to launch Diffle.com and start integrating games in with it. I'm hoping that Bootstrapacitor is something that'll speed things up and give us publicity, but if it flops again, we may be better off just concentrating on the games and pushing that out.

yCombinator

The yCombinator responses came on Tuesday, and...we didn't get in. I have a bunch of suspicions why, but ultimately it boils down to them only being able to interview 30 groups and us not being one of them.

I was feeling pretty bummed for about an hour, then I read some of the comments on news.YC by other rejects. And I had another idea for a startup. The biggest part of yCombinator's value proposition, for us, is the ability to work in a high-energy hacker environment, where we're continually getting feedback from our peers. And that part is not limited to yCombinator: we could easily organize groups of startups together and have some friendly competition going. We just need to group startups together.

So, I'm taking a little diversion from Diffle (hopefully not more than a week or so) to work on RejectedByYC.com. It'll have forums (which I've already setup), "demo groups" of about 10-12 startups, feature announcements, quick feedback, and karma. The hope is that this'll provide motivation to many teams who were not able to get into yCombinator or other funding, letting us maintain a high level of energy even though we've been rejected. Plus, by grouping them together, we may give them a brand advantage that lone startups don't have.

PIL

Lest I fall into the forget-about-updating trap again, I'm going to post an update about the past 5 days or so. I'll enter these as 3 separate entries as they're really 3 separate topics, though in reality I'm typing them all out between compiles at work.

On Saturday, I setup PIL, the Python Imaging Library. We're using it for thumbnails. Unfortunately, PIL is a real pill to install. I must've spent close to a day on it, Googling for everything I could find and looking through several config files.

In the end, my problems were due to 3 problems:

1. PIL requires libjpeg to operate on JPEG files. And the Debian package is not sufficient; it must be downloaded and built from source
2. libjpeg must be installed with the "make install-lib" command. The standard make install doesn't cut it.
3. PIL's setup.py will tell you that it successfully installed JPEG support even when it didn't.
   

Unfortunately, problem #3 masked problem #2, which led to a long wild-goose chase. I only found the problem when I tried a "python setup.py build_ext -i", which fails properly if libjpeg is not correctly installed. It at least told me that the header files were missing, which led me to read libjpeg's README file more carefully and fix the problem.

In my Googling, I found that many other people have had the same problem, and the PIL developers have been very unhelpful in resolving it. So once I make this public and Google indexes it, perhaps it'll prove useful to folks.

Month's catch up

So, I haven't updated in a month. I suppose that's good, since it means I've been working instead of writing about working. Though I imagine it's rather frustrating for the readers.

I won't even try to go over the features I've implemented. Judging from the last entry, it includes, comments, comment threading, tabs/subpages for user and game pages, AJAX ratings, comment replies, private messaging, new sort orders, plenty of bugfixes, and probably some stuff I've forgotten. I'd have to go over the Subversion logs to keep track of it all.

There are two developments that I want to include, since they're not in the Subversion logs.

We got our dedicated on March 13. It's a bottom-of-the-line Celeron - no sense spending more money than we have to if it never takes off. IIRC, Mike made the order from GoDaddy on a Tuesday, we had it by Thursday night, and that's when I started setting it up. The default firewall installation blocked DNS, so we thought that the DNS hadn't propagated when it was really my own stupidity.

Even before the domain worked, I started setting it up. Made a few subdomains for dev tasks and setup Apache virtual hosts for them. We're *only* using Apache for development & administrative tasks; the main production server is Lighttpd with an SCGI connection to Python. But I didn't want to figure out PHP/Perl/WebDAV for Lighttpd and I figured it'd be good to have our tools in a separate process for security/performance reasons anyway.

Over the next couple days and the weekend, I setup user accounts, upgraded Python to 2.5 & installed libraries, installed Lighttpd and configured it, configured flup and wrote a quick launch script, learned how to write /etc/init.d scripts to start everything, and basically got things working. By the conference call on Sunday, we were mostly good. I think I needed to do a couple tweaks to SCGI and flup to get the app running, and it wasn't until the next Wednesday that I installed monit.

Speaking of which, there were a few useful utility programs we rely on heavily:

1. Pligg, for organizing all our links. HOW-TOs, tutorials, documentation, 3rd-party download sites, potential game developers we want to approach, and competitors all go here.
2. MediaWiki, for documentation. We aren't really using this yet.
3. AWStats, for web server statistics. I was surprised how easy this was to setup.
4. Munin, for monitoring server health. Ditto
5. Monit, for automatically rebooting services when they go down.

Those were all part of the setup process on Friday/Saturday/Sunday.

The other interesting development was that we applied to yCombinator. I had previously floated the idea, but we weren't terribly interested then. However, many of our objections were answered by the ensuing blogosphere buzz (SFP applications are apparently a big deal in the startup world), so I asked my cofounders to give it a second look. Thus followed about 4 days of scrambling to get all the information we needed to fill out the application.

Then PG e-mailed us back and said we had too many weakly-committed founders, and probably too many founders in general.

I had misinterpreted the deadline, so we had an extra day to revise our app. We had an emergency conference call that night, which was probably one of the hardest and most awkward conversations I've ever had. The three founders who were not willing to commit full-time to Diffle were cool about it, though, and we ended up deciding it was okay for me and Mike to apply as a duo, and adjusted equity shares appropriately.

We're still waiting to hear back from yCombinator. Obviously, I hope we get in, since they add a lot of value. But we're going to go ahead with this anyway, even if we don't. I want to see this product exist, and we can continue to do that even if yCombinator doesn't want to invest.

Right now, Mike is working on the layout with Matt's help. I just finished adding comment replies and logout, so I've got a few cleanup items left. Then I want to move towards writing actual Flash games and starting on the game creation engine.

Database libraries

Monday - Wednesday was spent redoing the database access layer. I'd been using web.py's built in database libraries, but that's not suitable for the production site, because it uses a global database connection. If we need to move to multiple databases, it'll be a royal mess.

I considered using SQLAlchemy and spent much of last week researching that, but IMHO it's overkill. I don't need a full object/relational mapping layer; I'm quite happy with SQL and don't need to write my SQL statements in Python. So I ended up writing a thin wrapper on top of raw DBAPI classes. It basically provides convenience methods similar to PEAR DB for querying (using SQL), along with direct SQL generation for INSERT, UPDATE, and DELETE.

Am very grateful for unit tests. They let me change everything globally and let me know instantly where I'd broken things. Just need to flesh them out; a couple pages don't yet have tests.

This morning, I got a comment box to appear on the game page. Wrote the database schema for comments on the subway. I'll integrate it in and write the code when I get home.

Game page

Got the game page working, and hit counters. That didn't take long. Still no comments, though.

Last week's activity

A full week between posts. Ah well, shows I've been spending the time hacking rather than blogging about it.

Last week's conference call went well, despite not having something demoable. Matt had lots of good logo choices, though he was going to keep working at it and see if he could come up with anything better. I think it was last week that James finished the registration page layout. And we had some incorporation & decision-making stuff to talk about.

When I left off, I was incorporating the mock layout into the real app. That went smoothly - I just copied over the DreamWeaver HTML, CSS, and image files, replaced the content and login boxes with template vars, and it just worked. It's horribly messy CSS - DreamWeaver uses absolute positioning for everything, so I'm eventually going to have to go through and redo the format with a sane box model. But that'll wait till after the logo and final layout, since there will likely be further tweaks. This is good enough for testing & demos.

I also had to change the formatting a little to get the login box to fit. Not entirely satisfied with it; we'll likely have a space problem if we add OpenID. But we can cross that bridge when we come to it too.

When I got to the registration page, I realized that I really should figure out internationalization issues first. So that was most of this week. Started by simply trying to setup a null translation and load the resource bundle based on a database field; I finished that on Tuesday (yay for Monday being a holiday). That didn't work when I tried to add actual messages, so I spent some time Googling around for gettext tutorials, didn't find any, and then finally looked through the Mailman source. Fixed it on Thursday, just after Grey's Anatomy (I seem to get a lot done during that show. Maybe it's because last week's episode had me wishing that they had killed Meredith.)

I started working on the validation libraries concurrently. Wrote out the basic design on the Red Line Wednesday, then came home and typed most of it in on Wednesday night. Added the unit tests on Friday, finished up the validators, then spent all day yesterday integrating it in with the registration form and debugging. Seems to be working fairly well now.

I also did some minor CSS tweaks so the registration and upload forms look relatively pretty now.

Now I'm trying to finish the game page (instead of having it link directly to the SWF file), and hopefully get to comments. Yeah, I said that last week, but i18n and validation intervened. I'm glad that we're building a really solid app first, though, and then adding features. It gives a much more realistic idea of the schedule, and we can always cut features for launch.

The current plan is to launch with basic game uploading, commenting (threaded, with Markdown formatting), rating, and nothing else. That'll hopefully be enough to keep visitors coming back, and then we can improve it rapidly, add features like studios and the game creation engine, and listen to user feedback.

I'm thinking sometime around April for the date - if we can time it for spring break, that'd be great.

Catching up on testing....

I've been delinquent about keeping up with this. You'd think that with simple few-sentence updates, I'd remember to keep a log, but no. I'll have to reconstruct much of the past week from memory.

So, when I left off, I was just getting Emacs running again with VMWare. My next priority was testing.

I setup a MakeFile with some really simple phony targets - one to clean out all the .pyc and ~ backup files, one to run the built-in server for development, and one to run all the unit tests. I think this was Friday or Saturday a week ago. I do remember that I had no demo for last Sunday's conference call - I was still working on testing and refactoring.

Testing was a little tricky. I had to figure out twill - I think that was mostly last weekend, and the week before last. I'm beginning to love Python's dir() and help() functions - there's no complete API reference twill, but it has great docstrings. From there, I had to integrate it into doctest (I punted on unittest because I really hate xUnit-style tests; they require so much boilerplate code that I find myself ignoring the unit tests anyway). Never did get doctest.testfile() working; instead, I wrote a script to walk all the source dirs, turn the Python file into a module name, dynamically import it, get a reference to the module via globals().__dict__[modulename], and pass it to doctest.testmod(). I also wrote a twilltest helper module with some assertion functions that I wanted to be able to use for tests, and then added it to the doctest scope via extraglobs. This was done throughout most of last week; I wrote the helper functions as I needed them for specific tests.

After the meeting last Sunday, I spent the day coming up with usability personas. I think they're a good thing to have; as we design new features, it's nice to be able to say "David needs this" or "Katie won't use the site if we can't do this." They've already helped illuminate some conceptual muddiness.

Last weekend, I also moved things around so that all controllers live in one directory in the www directory, and pulled lib (helper Python files) and templates (HTML) back to the top of the trunk. The main entry point (main.py) and the unit testing script (runtests.py) were moved all the way to the top level. Still not sure if this is the right organization; there will likely be further changes as the program grows.

While I was at it, I had the request dispatcher auto-detect controller classes (through looking in www/controllers) and construct the URL list itself (through looking at the exported url_pattern global). My svn log says this was last Tuesday, 2/13. Python's dynamicity is fun.

You would've thought that with Wednesday being a snow day, I would've gotten more done, but no. I was working on stuff for my real job (from home) almost all day. And then on Thursday I had to shovel. Finally finished the unit testing framework on Thursday night, while watching Grey's Anatomy, and these last couple days have involved writing more tests for front page, logins, sessions, etc.

Am working on integrating James's rough cut at a layout now. If I get that done, I may take a crack at comments.

VMWare & Emacs

Got over my Emacs-can't-select-text-with-the-shift-key problem (I'm using CUA mode, otherwise Emacs wouldn't select text with the shift key anyway). Turned out it was this issue, and I just had to disable the ATI Radeon taskbar icon. Who'da thunk it?

Testing, testing

Didn't get nearly as much done this past couple days as I hoped. I wrote some simple Twill scripts and tried it out from the interactive interpreter. Read up some on doctest and unittest - I already use the former, but really dislike the latter because it's a jUnit-style framework (some other time, I'll write up a blog post about why that sucks). Also installed BicycleRepairMan, PyChecker, and some additional Emacs modes, though I haven't tried them out yet.

I also had to deal with some Windows idiocy and re-copy my VMWare virtual disk because VMWare has trouble when the disk file is owned by some other Windows user. I had created the VM initially as AMHERST/jdtang - yes, 2 years after graduation, my computer still thinks I was on the AMHERST domain because Windows apparently caches logins (good thing, too, otherwise it would break when I went home). After wiping my computer, no more AMHERST for me, so I created a local account that had trouble writing to the logs and virtual disks created by the domain account.

Intro post

[Original blog description: "For posterity's sake, this records the very early days of Diffle Inc. (or so we think it'll be called) and its Flash-based social games site. It's intended to be private, at least if/until Diffle manages to become successful."]

I want to create this blog for a couple of reasons:

1. So I can better estimate how long new features will take. It's amazing how easily memories of implementing some feature get distorted.
2. Because I find it fascinating reading about the early days of businesses, particularly businesses that later grow enormously successful. I can't guarantee this'll be enormously successful (I rather doubt it, actually), but just in case, I'd like to record things.

This is really more of a journal than a blog, since it's intended to be private unless Diffle (or whatever we decide to call it) gets big. I've got a bunch of reasons for that: I don't want to get in trouble with my present employer, I don't want to set up expectations that we can't meet, I don't want to give away secrets that'll help potential competitors. But mostly, I just think it's premature and a distraction from actually getting the site implemented. So this'll have entries on current events, but it's not really intended for a readership. Expect some chop.

Anyway, the story so far:

Diffle is supposed to be a website for social networking based on simple, 2D flash games. Kinda like a cross between Yahoo!Games, MySpace, and Fanfiction.Net. We've actually got two names under consideration: Diffle and GameClay. The current plan is to launch with Diffle as a pure social networking site centered around games. If it takes off, we add GameClay as an additional feature. If it doesn't, we abandon the diffle.com site and rebrand as gameclay.com with the additional game-creation features.

My involvement is 2.5 weeks old at this point. The site initially started with Mike's summer internship, researching companies in the space, and his getting together with James. Andrew was brought on board for finance/legal, and Matt for art & graphic design. They brought Xin on board as their programmer around late October or early November.

That was the status when I saw Mike at Homecoming. We chatted a bit, he asked me if I wanted to do some Flash development or front-end work, but it would've been a contract position rather than a founder stake (with 5 co-founders, the equity is already spread pretty thinly). Besides, I don't know Flash.

Fast forward to late December. Xin's on a student visa, so he finds out that he can't co-found a startup without putting his visa status at risk. There's a mad scramble to find another programmer. As I understand it, there were a few people ahead of me on the list, but none of them wanted to do it (hey, 4 guys a year and a half out of college starting a business in a hot sector, I'd be skeptical too).

Mike contacted me over AIM on Friday, 1/19. He e-mailed me the business plan, and I took a look at it as soon as I got home from work. It looked very exciting; there's a potentially big (though risky) opportunity here, and it was exactly the sort of project that I've wanted to do since college. Mike and I had a good conversation on Saturday morning, we both were very enthusiastic, and so I was brought on board as the programmer. I spent most of Sunday setting up my development environment - Linux, Python, MySQL, assorted libraries. Luckily, I have a spare Debian VMWare image for just such purposes.

First Skype conference was on Monday night, and I got a quick introduction to what had been done already (a lot on the design/conceptual side, essentially nothing on the technical side) and where we were going. Spent most of the rest of the week learning web.py. I also got permission from Tosin (NOTE's current head at Amherst) to use one of the NOTE servers for prototyping/demo and install whatever I wanted on it. For the next week's conference (1/28), I had a really simple demo with a barebones front page and registration form up.

The week after that, I got a lot more done featurewise. Finished most of sessions and authentication during the workweek - I usually managed to put in a half hour or so before work while eating breakfast and an hour or two after work. Plus, if there are sticky spots, I write 'em out on paper and pencil on the Red Line (much of session handling was done in a day or two in this manner). The demo on 2/4 included sessions, logins, the ability to upload games, games appearing on the front page, and a simple user page. Uploading was done all on Saturday, and having them show up was Sunday morning. Still far from complete functionality, but good progress.

I said that I likely won't have as much done for next week, where "not as much" may mean "nothing" featurewise. Instead, I'm cleaning up the nascent code base and investing in some infrastructure requirements. I setup a Subversion repository on Monday night after I finished watching 24, then posted to web.py for testing suggestions and started checking out twill yesterday. I'm hoping to write some tests tonight, establish a framework for testing, and do a one-step build process that handles pychecker and all our tests.