Devops Drop 014

The Last Lean Startup Bundle: 48 hours to claim $3,000,000 in prizes

http://www.startuplessonslearned.com/2011/09/last-lean-startup-bundle-48-hours-to.html

One book 750 worth of stuff.  the 5 and 10 books kinda not so good...

HP Launches Private Beta Of Cloud Compute, Storage Services

http://www.crn.com/news/cloud/231600974/hp-launches-private-beta-of-cloud-compute-storage-services.htm

started a twitter discussion.  I pointed out that I am pretty sure (certain) that they did not use opsware as a the deployment tool of choice... 

meta point is that second gen CM tools are dead.. Tivoli, Bladelogic, Opsware... #puppet and #chef rule the cloud... no turning back... 

Fault Tolerance and Protection

http://www.kitchensoap.com/2011/09/08/fault-tolerance-and-protection/comment-page-1/#comment-8339

GN&C FAULT PROTECTION FUNDAMENTALS

https://pub-lib.jpl.nasa.gov/docushare/dsweb/Get/Document-316/08-031+GN%26C+Fault+Protection+Fundamentals.pdf

Thinking outside the box... 

-----------------------

Autometrics: Self-service metrics collection

http://engineering.linkedin.com/52/autometrics-self-service-metrics-collection

lnked in system 

uses zookeeper for coordination. 

Kafka 

• 500k+ metrics collected in a production data center every minute or about 8800 per second.
• The average number of metrics per service is about 400, although some services have thousands
• 1 minute resolution is maintained for 30 days, 5 minute for 90, 2 years of 1 hour resolution.

Amazon deploys every 11.6 seconds

Jon Jenkins

Amazon.com

 [e-mail]

June 16, 2011

http://www.youtube.com/watch?v=dxk8b9rSKOo

http://news.ycombinator.com/item?id=2971521

some gems in here.

nov 10 2010 all amazn web servers went ec2... 

Makes a great argument for utility computing (i.. cloud)

Apllo deployment system.  

one moth stats 11.6 depolys per second, peak 1079 in one hr,

10k avg sim deploys

30k peak. 

But even their tooling reflects decoupling. Every tool follows the self-service model ("YOU do what you WANT to do with YOUR stuff"). Their deployment system (named Apollo, mentioned in the slides) and their build system, and their many other tooling, all reflect this model.

Cons. What happens is that you might be reinventing the wheel at Amazon. Often. Code reuse is very low across teams. So there's no shared cost of ownership at Amazon, more often than not. It's the complete opposite at Google w.r.t. code reuse. There are many very high-quality libraries at Google that are designed to be shared. Guava (the Java library) is a great example.

Another con. You may not know what you're doing. But as a team you will still build a rickety solution that gets you to a working solution. This is the result of giving a team complete ownership: they'll build what they know with what they have. Amazon is slowly correcting some of these problems by having teams own specific Hard Problems. A good example is storage systems.

And a lack of consistency is a common issue across Amazon. Code quality and conventions fluctuate wildly across teams.

Overall, Amazon has figured out how to decouple things very well.

-----------------------

Data scientist: The hot new gig in tech

http://tech.fortune.cnn.com/2011/09/06/data-scientist-the-hot-new-gig-in-tech/

The gig which requires the specialist to capture, sort, and figure out what data are relevant is one part statistician, one part forensic scientist, and one part hacker.

A recent report from the McKinsey Global Institute says that by 2018 the U.S. could face a shortage of up to 190,000 workers with analytical skills.