As one of the most recent additions to Digg’s engineering team, I thought I would share with you some of what it’s like to work at a place that’s doing some really innovative and exciting things on the Web. When I joined Digg a little over six months ago, I had no idea what to expect. Coming from an organization of 15,000+ employees at Yahoo, you form certain expectations. Startup life is an entirely different animal in terms of speed and agility and one of the key lessons I’ve learned is that you must strike a balance between engineering and business. My first few weeks of joining Digg, I began watching and tediously studying what makes the engineering organization tick. Culture plays a large part in how engineering works with other departments such as operations and product. The culture at Digg is great and I can’t stress that enough. Getting to work on hard problems is what every great engineer strives for. We want to be challenged and pushed to our limits. I’m excited to say that I joined a company that allows me to do that.

Operations vs Engineering

In many companies, you have the engineers and then you have the operations team. Engineers complain that the operations people aren’t nimble and agile. The operations people complain that the engineers never plan or keep them in the loop until the last minute. In many ways it’s an uphill battle for most organizations. Because I deeply care so much about infrastructure, it forces me to play a dual role. I have to understand what’s involved from the engineering effort, and what’s required to fulfill the needs of the operations team to keep things moving at light speed. One of the great things at Digg is that the two teams work closely together to achieve the same goals. Respect and guidance is key.

Under the hood

We’ve been making a lot of changes internally. Starting with providing more ownership to engineers, and investing in infrastructure that will scale and allow us the agility to make rapid decisions. A few months ago we moved the entire Digg codebase from Subversion over to Git. Ultimately we saw the headaches of merging in subversion and we had enough engineers already using it on open source projects, that we knew it was the right direction to improving our foundation as a company. Working at Yahoo, I quickly learned how valuable distributed computing, storage and logging can be for an engineering team to function. At Digg we have been been making investments into our data storage infrastructure (Hadoop & Cassandra), along with building out massive logging infrastructure that allows us the ability to collect metrics and reporting in real-time. The core infrastructure team at Digg has been using Hadoop, and our logging framework to mirror production traffic against our test clusters to analyze performance and do internal dark launches. Performance is critical to engineering and our cost of operations. We’ve been working hard on eventual consistency between multiple datacenters, multi-tier caching, front-end and CDN performance, automatic failover of services across datacenters, and building a new software stack that isn’t just scalable and fast, but is also cost effective and flexible.

Open Source

Digg’s support of open source has been truly refreshing; considering at other companies I’ve worked at it has proven difficult to get patches or new projects released. Not many companies realize the true value that open source provides. VP of Engineering, John Quinn, saw this early on. I was really happy to see someone on the executive management team actually understand its importance. It was one of the many reasons I joined Digg. In terms of open source at Digg, we are heavily involved in the community. We understand its importance to our infrastructure as well as how important it is to give back to the community. We have developers working on: Cassandra, Thrift, Scribe, Hadoop, Hive, and memcached to name a few. Along with a slew of our own projects over on Github.

Bringing it together

Without a solid foundation to really build upon, you’re left spinning your wheels on cruft, frustration and telling the product people, that new addition or change is going to take weeks to months, instead of days. I like thinking in days, and sometimes that surprises people. When your tools are automated and your foundation is solid, it really allows you to iterate, test and actually make a decision. We’ve all sat in meeting after meeting trying to come up with ideas and decisions of what to do, just to setup up another meeting. You have to have a balance of risk, foundation, culture, open source and most of all, talented people who can see the big picture.

We have many hard problems to solve here at Digg, and many more amazing ideas to build on. We are making remarkable strides in just the few months that I’ve started working at this company and I look forward to the road ahead…