Now that you have you system and project environment all setup you probably want to start developing. But you likely don’t want to start writing an entire project fully from scratch, as you dive in you’ll quickly realize theres many tools helping you build projects and sites faster. For example making a request to a website there’s Requests, for handling processing images there’s Python Imaging Library, or for a full framework to help you in building a site there’s Django. With all of these there’s one simple and common way to install ...

As the web evolves, Google’s crawling and indexing capabilities also need to progress. Weimproved our indexing of Flash, built a more robust infrastructure called Caffeine, and we even started crawling forms where it makes sense. Now, especially with the growing popularity of JavaScript and, with it, AJAX, we’re finding more web pages requiring POST requests -- either for the entire content of the page or because the pages are missing information and/or look completely broken without the resources returned from POST. For Google Search this is less than ideal, because wh...

The Ruby language is beautiful. And I think it deserves to break free from the Web. I think the future of Ruby is firmly stuck in Web development, though, so I’m going to invest in a new language for data analysis, at least for now. This is a look at the fantastic language I came to from Java and a look at a possible candidate. (Update: I’ve since written a followup.)Java to RubySix years ago, I added Ruby to my technical arsenal. I learned C++ and Java in high school, and I planned to use them for data analysis in college—mainly for research kung fu. But when I disco...

We live in a polarizing society, so perhaps it’s inevitable that our experience of inequality should be polarized, too.In the first place, there is what you might call Blue Inequality. This is the kind experienced in New York City, Los Angeles, Boston, San Francisco, Seattle, Dallas, Houston and the District of Columbia. In these places, you see the top 1 percent of earners zooming upward, amassing more income and wealth. The economists Jon Bakija, Adam Cole and Bradley Heim have done the most authoritative research on who these top 1 percenters are.Roughly 31 percent started or manage ...

There are a lot of differences between programming, and programming professionally. The most notorious of which, is deadlines.DeadlinesWhen you're writing code for yourself, you can spend as much (or as little time) on it as you please--but when you're writing code for other people, you've got only a limited amount of time and resources to get the job done. In my experience, this typically leads to one of two situations:You've got to extend the deadline to finish the job properly.You've got to write some dirty hacks to finish the job.If you've ever done professional programming, you know what ...

Admitting that may be career suicide, or possibly it will cost me dearly because 'software engineers' are raking in the big bucks these days, but the fact of the matter is that I'm a programmer. It's what I do best and it is the job title that I associate with most because it feels as though the biggest chunk of me will always be most likely to blurt that out when people ask me what my job is. That I like to program definitely helps.So instead of choosing some fancy title for what it is that I do because that might possibly make me a bit more money I chose to embrace it, and I have yet to regr...

The difference is subtle, but it is a big difference. The ON condition stipulates which rows will be returned in the join, while the WHERE condition acts as a filter on the rows that actually were returned.Simple example: Consider a student table, consisting of one row per student, with student id and student name. In a second table, a list of grades that students have received, with student_id, subject, and grade. Give me a list of all students, and show their grade in Math. This requires a LEFT OUTER JOIN, because you want all students, and you know that some of them didn't take Math. Here a...

The last couple of years more and more people have been moving towards Agile development. These techniques aren’t new, most we’re devised in the 80s or 90s. But finally these days programmers and (more importantly) business consultants, architects and clients have learned to love and embrace Agile development.Evolving requirementsIt has now become common knowledge that you can’t write down all the requirements before you start the project. These requirements have to be written down in an evolutionary matter. In short cycles/sprints we build pieces of the pro...