Monday, January 31, 2011

The value of a University Degree in Computing

I got a piece of paper that says I know what I’m doing in computer science and another piece of paper saying that I know math. Today, I want to talk about the value of those pieces of paper as it stands today.

Ultimately, the value of a university experience is multi-dimensional and varies person to person. I’ll discuss multiple vantage points and weight my thoughts on the value.

Computing Resources

Back in the day, you had to be at university to even get access to big computing resources. Now, with cloud computing, you can get more computing power than what I had access to for less than your lunch. So, don’t go to university if you need awesome computing power; you can pay for it with a job. That being said, some universities have ridiculously cool stuff like 3D scanners. However, there is a hacker right now using a kinect and a Markov process to build a precise 3D scanner using their Lego Mindstorms kit. It may be worth going to university for equipment if the equipment is extremely advanced, but otherwise don’t.
  • -1 University
  • +1 Cloud Computing
Exposure to Mathematics/Computing/Material

With the internet, you can get a ton of exposure to mathematics. It’s all there. You don’t need a special library card to get it. That being said, there are some pay-walls for springer/ACM and other places with cutting-edge libraries. It’s worth going to college to be able to SSH in and then wget an article’s text. Otherwise, master Google (or stack overflow).
  • -0.8 University
  • +0.2 Cleverness
  • +1 Internet
Exposure to Breadth

Where do you begin to learn and teach yourself something? How are these seemingly unrelated things connected? University wins big by having a structured series of classes that in theory build on one-another. An ideal university will cover basically everything that is at the foundations and give you a good sense of the field.

However, you can get the same thing by reading their course catalog and degree requirements and having the discipline of going over the material yourself.

  • +1 University
  • +1 Discipline
Mathematical/Engineering Mentorship

Having access to math is one thing, but having the decoder ring is another. As a budding graduate student, I found that listening to a lecturer was valuable in not learning the material but rather how to interpret/construct the material. It is one thing to know that something exists or what something is, but it is entirely different thing to understand how something was constructed. The side-notes/annotations a lecturer gives on how to prove a theorem rather than the formula.

The same was true for a handful of computing courses where you got insight into the design process rather than “here are some facts, now go”. Problematically, this is a very hard thing to teach since this kind of knowledge is usually lost over time.
  • +0.5 University
  • -0.5 Sucky Professors
Mentorship

This is the big win for university. If you don’t work under a professor as an assistant, then you are really missing out. There is a saying “Those that can, do; those that can’t, teach”. And, it is very true for many instructors, but the other half are off in their zone mastering a part of the cosmos that they own. You can learn a lot from these professors, and you should listen to them and have an open mind.

Most students fuck this bit up and claim professors are out of touch, and they ignore the oddities of their professor rather than studying them. If you study them, then you can learn a great deal about smart people.
  • +1 University
  • -1 Closed Minded Students
Peers, Colleagues, and Professors
You write a blog post and no one leaves a comment. Did it get read? Probably not. If you work in a group where people care about their grade, and you control a piece of it; is it going to get read? yes. If you get a paper back with tons of red ink on it, then it got read. If another person wants to put their name on your paper, then it got read and you got feedback.

The nice thing about university is that it provides a feedback mechanism for how you interact with the world. Granted, we are transitioning collectively to a more casual style of writing.
  • +1 University
  • +1 Casual Writing Style
Teaching

Where else can you get 30 students handed to you and then have the opportunity to teach them something. I went into teaching, so I could break my fear of public speaking. I don’t have that fear anymore.

When you teach a group of students, you learn a few things about what is and is not effective. Mostly though, you learn you can’t force knowledge onto people. Instead, you can only open the door and let them study you as you walk through it.
  • +1 University
  • -1 Students (the poor bastards I iterated on)
“Other Stuff”

Anything outside of your discipline was required to be an “educated” or “civilized” man. I’m referring of course to when women were not allowed in university. Now of days, with the internet; you can get everything you missed in humanities and then some using the internet. Beware of Trolls.
  • -1 University
  • +1 Trolls
Campus

Generally speaking, the campus is a dream for extroverts. Introverts are fucked. The nice thing about campus is the ease of access to smart people. You want to bug your professor, go for it. Most of them like it.

Most students fuck this up by not bothering their professors because they are too lazy to get out of bed. I had fairly insane office hours (every day including weekends, 8+ hours) and I welcomed many students. I had a lot of people fail that I never saw. I had 2 people actually take advantage of my hours; I turned one of them from a D student to an A student.
  • +1 Extroverts
  • -1 Introverts
  • -1 Lazy Students
  • +1 Humble Students
Big Finish

Here is the question: Where do you want to be?

I think university can make it easier to get for sure, and it doesn't have to be an ivy league either. However, most students are going to waste their college career because they just don't know how to extract real value from it. It can be a lot of fun, or it can be a growth period.

Do you need it now of days? I don't think so. All you need is an employer that is willing to push you. Keeping in mind of course, the employer is going to push you in a direction that is relevant to the company. Some companies, like Google, know that if they enable the herd to stray, then you can capitalize on that knowledge and let it integrate into across the entire organization. Most companies are not mature enough yet to take this path.

If you are stuck and the company is pushing you in a direction of a cog, then I recommend getting efficient at that direction and explore side projects on your own to bring up your knowledge and skill set.

As a rule of thumb, I don't recommend letting a company control your career. You should be in control, and you need to realize that you are competing in a global work-force. Technology moves fast, and you need to move faster. A college degree is ok, but university is getting efficient at churning out mediocre developers.

I think the four years at university can be replaced with one year of ultra-hardcore mentor-ship. I'm talking 110+ hour week of hard-core work, education, and side projects. When I figure out how to make it legal, I'll announce my program.

Saturday, January 29, 2011

CouchDB rocks, so I'm going to make it relational.

Here is the plan, and I'm soliciting for help to start planning and building an open source project.

About two years ago, I build my own programming language called Kira to solve "the scalability problem", and I was looking at how a schema combined with how it was used (via my own LINQ) to study patterns. Needless to say, that studying those patterns were fruitful in discovery of the emerging NoSQL solutions. You can model each NoSQL solution using a relational DB, but you either get ease of scale or performance. My interest was starting at a purely relational model and then solving the system using NoSQL systems.

I want to say something: this problem is pure geek crack. Unfortunately, my study was pushing me more and more away from SQL and into NoSQL. Here are the things that I can build anything with:
  • S3
  • Redis
  • CouchDB
  • node.js (+ otto)
Now, there is a certain elegance to SQL from a support point of view. When you build something from a NoSQL solution, then you are indebted to build a management layer for what-ever you built. You tend to always have to do this, but NoSQL forces you too early on in the development process. While this is a problem for many businesses now, it is opportunity for hackers/geeks to build something awesome like phpMyAdmin and n00bify NoSQL.

Right now, I think CouchDB is ultra awesome and ultra easy. However, it has some business problems that I think are naturally solvable via node.js (due to JavaScript portability).

I'm solving a couple with otto and vacuum now, but those are primarily experimental. I would like to do something a bit more robust and general.

Saturday, January 22, 2011

How to be a hacker?

I get asked "can you teach me to become a hacker?", and I'll be honest, I have no idea. Asking me puts me in an uncomfortable state of cruelty where I say "just got to work hard at it and not be a n00b". This got me thinking about how did I become a hacker?

First, what is a hacker? Well, I'm going to be general and extend the concepts of a hacker beyond computers. I think hacking is the ability to see the system in place and the willingness to exploit the system to make it your bitch. There are many systems in the world, but computers are easier to hack because the time to fail is so quick. Hackers are not confined by the boundaries of computing. I've met people that hack people over the phone (really good salesmen). I've met people that hack the human body (doctors and medical enthusiasts). I've met people that hack the fear of death (preachers).

I've thought about writing a book called "Born to Hack" inspired by "Born to Run" as I believe the two are related. We are here today with all the flashing lights and buttons because we hacked reality and made it our bitch.

We are the people who hack, yet some of us don't. So, I'm going to give three simple steps that someone who doesn't view themselves as a hacker to try.

Pick a medium

I chose computers because they are shiny, and I have no sense of patience. I can't wait a week for cultures to come back, so I would have failed at medicine. The key here is to pick a medium that you can readily manipulate and measure effects.

Work really hard at making the medium your bitch

Here is where you just work at it. If the massive neural network in your head is good enough, then you will make it work. You will understand the system, the rules governing the system, ways to manipulate the system, how to break it, and how to fix it.

Do something new

This is where you must see something as a goal and work towards it. Everyone I've met is a genius in hindsight, and the trick is to realize that everything can be made better since nothing is perfect.

The entire process takes either 10,000 hours/ten years give or take 5,000 hours/five years. The key to being great is waking up everyday and working towards your goal. I code every day. What do you do everyday?

After all, you are what you do.

Wednesday, January 19, 2011

node.js production environment #63

I pulled out the "production/" scripts from the WIN and rebuilt it as simplenode. Basically, simplenode is a very simple init.d script and a forever loop that enables
  • per restart logging (can fill up server)
  • crash throttling (5 seconds)
  • restart on reboot
All of this takes place /var/nodejs

Now, I'm thinking about making a patch for node.js makefile such that it will install everything it needs into /var/nodejs so I can chroot it. Has anyone else done this?

Places to improve:

As of now, I'm logging everything so if I dump too much diagnostic information, then I'm going to get screwed later on. This is fine for now, but eventually, I'll need either to delete logs, truncate logs, or export logs to S3... This gives me an interesting idea for a service, ;)

The five seconds is arbitrary, but I need a way to detect if the script is in a "bad place" and the state is corrupt. I've had this problem with using the file-system since a crash botched up the state, and I didn't respond in time and the forever loop was making the entire system crawl.

Saturday, January 15, 2011

vacuum.js alpha (slightly inspired by nature/biology based computation)

I just cut a very rough version of vacuum.js to test it out with a couple of friends.

One thing that I miss from RDBMS land is the ability to fix bad data using command line SQL. For instance, with this code

update places set state=some_complicated_function(state) where state='blah';

I can fix bad data that somehow got into the system. A consequence of moving fast is that you get unexpected data (that may have more value than you think) that can break. Either you plan it all out or you let some criminals in the database to see how the business case works out. Sometime, a lack of planning enables sales to hack things out on their own. ;)

Anyway, back to vacuum and CouchDB. There isn't a way to update the entire database or a subset of the database easily; the easiest option you have is to build a view that brings out bad data and then a cronjob that cleans things up every now an again.

That's vacuum.js alpha.

My example data-set that contains a bunch of US state abbreviations. Except, some are very bad and malformed. In RDBMS, this is fairly easy to fix. With vacuum.js, its even easier to fix.

The current example provides two functions that run in the CouchDB context which marks documents as bad if they don't have a two letter state code (I could verify that it exists); the other function will try its very best to fix it.

The current version is a quick hack job, and I'm sitting down now to flush out a better version. As of now, its good enough to test and apply singular fixes for a couple of my project; however, since it polls, it is going to waste a lot of time on bigger systems. Check out the README to see where my current thoughts are.

As an interesting aside, I think NoSQL and thinking NoSQL design enables nature/biological thinking with MapReduce. I think it is very interesting to think of code working in an ecosystem where you can program receptors and bind actions to them. I may rename vacuum.js to something more in this vein when I'm finished with my plans. Using the same type of system, I could split/merge data with functional programming paradigms. I think this is the key to designing with MapReduce and CouchDB.

This is definitely going into the book I'm writing about CouchDB design.

Thursday, January 13, 2011

Barefoot walk in snow (Vikram five finger) testimonial

http://runkeeper.com/user/mathgladiator/activity/23264937

Oh yeah, 17 degrees out and 80% of the walk was in 1/2 foot of snow.

Do I recommend this to you? No.

"Ok, wait, what is the point of the post?" you ask?

Well, I like to push limits. I've been pushing my body throughout 2010, and I'm continuing in 2011. Cold is a personal study to me. Do you know how long you can last in the snow? In pants? In shorts? Naked? Naked in water? How long until you show signs of frost-bite or hypothermia? These are questions that I have answers too.

Sunday, January 9, 2011

PoTATo and PotATO with CouchDB

As many of you know, I'm having a love affair with CouchDB since it makes typical CRUD aspects of an application a breeze.

Last night, I posted my install on github:

https://github.com/mathgladiator/potato

In a nutshell, this is now how I install CouchDB in the cloud. I'll back-port any security lessons I learn into this script over time. If you agree to use this script, then you agree to the CouchDB agreement.

Saturday, January 8, 2011

2nd Law of Code Generation

I don't know what the first law is, but I'm basing the 2nd law of code generation around the 2nd law of thermodynamics. Basically, when you use a code generator, business value only decreases. That's why you should start at the highest level.

I'm working on some tools that generate code from documentation.

Friday, January 7, 2011

algebraic json differences (change logs, replication, oh my)

https://github.com/mathgladiator/jsondiff

This is a hacked-out version of an idea I've been playing with.

Since I've adopted CouchDB as my mind set, I would like to make life easier by focusing on JSON at a very general level. There seems to me a potential for some algebraic goodness with JSON. What does this mean? I don't exactly know yet. What I have so far is a non-commutative operator on a set of JSON objects. I've defined addition and subtraction. Multiplication doesn't make any sense, so I'm at the best I've got a group.

Use Case 1: Change Logs

I have an old copy of an object and now I have a new copy; I can use this code to compute a difference that will turn the old copy into the new copy. Ideally, this will enable audit logs as well as some form of revision since I can describe the entire object as a sequence of additions by starting out an empty object.

Interestingly Enough, if you design a special set of objects such that addition is commutative, then you get Google Wave and a nice level of concurrency (using an atomic journal).

Use Case 2: Undo, Redo

It's fairly easy to make a system to undo and redo changes to an object if you can study the changes abstractly from the outside rather than requiring a programmer to make notes of all changes.

Use Case 3: Replication and Optimization (REST/RPC)

If I use CouchDB and then replicate to a node.js server running Otto, then I can store a copy in node.js (or memcache) and then differentiate to produce an object that tells me how best to replicate to foreign services (like an RPC server you don't control, but you need too). Using duck-typing inspection, you can then send only the RPC commands that you need too.

Use Case 4: Algebra is oddly fun

I get joy every time I think "Oh, C++ operator overloading would make look this sooo nice". Unfortunately, I don't work in C++ any more and so I avoid that joy, but I get the joy of being able to ask questions like "Is this a group? Is it associative? Is it commutative? What is zero? What is equality?"; I'll write up my notes when I get time, but for now the code is live on github.

Concluding Thought, Ii think algebra plays a bigger role in computing than I originally thought.

Tuesday, January 4, 2011

cmdfu, roll your own heroku-like interface

Every year, I write something up in C to make sure I still have hair on my chest. This year, I wrote cmdfu. cmdfu is a simple tool for executing commands remotely in a secure fashion (using an HMAC-esque signing process).

You can clone it from its github.

I'm going to play to my strengths (i.e. not UI) this year and release some command line services like crondom.com and the yet unnamed node.js hosting service. cmdfu is basically the way I plan to offer up remote execution against my servers. There is preliminary documentation here.

Using bash magic, it is fairly easy to make your own heroku-like interface. For instance, I can make a crondom script to manage my account.
crondom create http://www.mathgladiator.com/cron/test/1minute.php

crondom ping every 5 minutes http://www.mathgladiator.com/ping.php

crondom list

crondom delete 1-341343
You may be asking, "why C?". Why not?

If you want to install, then you need to install libcurl. I should really make a real makefile, but for now I like what I got. If you would like to use it for real, then please leave me a message in my github account (or just fork it).