Coding ... Coding ... Coding ...

Coding .... Thats what we spent the whole day doing ... garage datacenter artist arrived at about 11:00am and we got stuck right into it. We began by preparing each of our own modules for integration, then finally putting it all together. We overcame quite a few issues and sorted out all our unknowns. Most of the components have been integrated now - (crewd) but all modules talking, we have been preparing for the last 3 months .... Im also abit tired now cause im not feeling 100% and also quite tired mentally ... 12 hours infront of the computer cant be good .... Yeh i think i'll make this entry quite short ...

My coding Quarters ...

OK ... Night ... going to sleep ...

Baby dont cry... Baby dont get ...

Some people are afraid to ask so they dont, then they complain about not getting ... its an easy concept though lots of people dont understand this ... including me until about a couple of years back.

We are grumbling and disgruntled in the office, unhappy cause we think we are underpaid, we wonder why we are underpaid and usually the answer its pretty easy, its cause we just didnt ask :)

I remember thinking a few years ago (possibly as a result of the dot com bust) that if i was offered a job i should take it and not question the money (yes i know ... i was dumb). The reason why i attribute it to the dot com bust is i graduated from uni around that era (2 years or so after) and the competition for jobs was huge cause no one was hiring. I remember when i was a grad it would be 500 grads to 4 jobs and if you were offered you should just take it.

Times have changed now and i have learnt alot over time. Now not only do i cry .. i also jump up and down, roll on the groud in circles and throw tantrums, so to speak ;) and yes it seems to work :P

At the moment im negotiating a 4 day week (doing infrastructure architect role *thank god*) with good contract rates and a great working environment. I am doing this so that i have time on my hands (albeit abit less) for my startup as time is drawing near. This deal is still under negotiations so lets hope my tantrums are enough to seal the deal ....

Chance Advice ....

Its funny how buddies of yours seem to creep up from nowhere. Guys that you were friends with before even starting highschool. You wonder where they are and all of a sudden you meet them at a the local department store, talk abit and found out that you pretty much are into the same things. Sometimes i ask my self the question. Was this a chance meeting or was it something pre-destined by a higher power. Anyway, whatever the case these chance meetings can end up really changing alot of things.

I had a buddy, lets call him Bedroom CEO. Bedroom CEO is actually doing a startup also, he lives (apparently) 5 minutes away from me (something which i wasnt aware of until a few months back). He is very in tune with the startup and technology scene here in the great land down under. Anyway, by a chance meeting we started talking and at about the same time we decided (in our own time .. not together) to do startups. Difference between Bedroom CEO and Garage CEO (myself) Bedroom CEO took the big step and totally quit his job.

I would like to do that aswell (i guess for a year) to get my startup of the ground but im in a bit of a different situation. If i did that, the repo's from the bank would be banging at my door and fighting hard with me to pry my house keys away as i would be kicking and screaming (yes im one of the sad *young* individuals who has a mortgage thinking property investment is the way to go).

OK since Bedroom CEO is very intouch with the Startup and technology scene here he has introduced me to various different events, various different reading material and various different people that provide information about startups, he is almost like the umbilical cord to silicon valley :P I guess it shows that even though i understand technology to a certain extent in the great scheme of the startup world im not even past the 9-month stage yet ;)

Last week Bedroom CEO scheduled some time with ryan junee from omnisio/youtube and we hungout and had some beers with various other geeky type guys and gal and just chatted away. We first hangout at the shelbourne for some beers, went to this startup/recruitment christmas party where there was some wanna be startup guys (us) and other startup guys later in the game who have been successful.

So here is a photo i took of that night (im not in it *as usual* ;))

So there you have it, i thought there was no startup scene here but apparently there is, i think its time for me to delve deeper into this world as im in the later part of my startup project (couple months to go till launch).

Catch you all in my next post ... (which im about to start in the next 5 mins :P)

Getting into the Software Side of things ....

CentOS

I guess its a legacy thing for me. It may not be the best *nix operating system for some people but i just want the system to work. In the past, both at work and at home i got very accustomed to redhat ... and now since redhat ES and AS are no longer free i had to find an alternative. For me CentOS was the most obvious choice.

I like CentOS because:

• Im pretty accusomed to how the Administor and manage the operating system, how to install the location of the various configuration and system files, startup scripts and standards etc
• It already had LVS and various other clustering software installed - along with its tools already installed
• Worked without any problems on the hardware that i purchased which was a big plus and all round easy to use OS - (for me)

MySQL

I used MySQL because again its something im very much used to. Used it during development at work and also when experimenting on my own and pretty much accustomed with how it works. There are alot of tools and libraries available. I also had very good documentation of the clustering packages that come with MySQL 6.0+ so it was the obvious choice to go.

The MySQL cluster now works over 4 servers and is bound together with a hardware LVS load balancer. Still getting used to operating the cluster commands but all in all its a pretty good tool.

CherryPy + Cheetah

Hmm okie, so why cherrypy .... I tried zope, django, turbo gears and at the end, after looking around and playing with the different technologies i decided to use cherrypy. I think its because things just came together alot easier with cherrypy and i was able to get what i wanted up and running alot quicker. I think its really just me here and possibly my old school way of thinking .... most of the other guys i know who are building sofware have gone down the django route.

Good thing with variety is later once our applications are finished we can share experiences and ideas so for me .. its cherrypy all the way ... :)

Python 2.5

It was easy to get my head around all the different concepts. The language was easy to learn and i could get applications built rapidly. Other then that nothing much else. I am currently using python 2.5 (which i complied specifically for the application). There is also 2.4 installed by default by centOS.

Lighttpd

I went down this path as a result of alot of reading and recommendations. In the past i have used apache (as most people may have) but after alot of reading and various types of experiments i have gone down the lighttpd path. It sets up pretty nicely and is performing well. I have tested its mod_proxy function but will also need to experiment with lighttpd's mod_fastcgi modules as well to see which one is actually better.

SQLObjects

When i was playing around with turbogears SQLObjects was the default ORM. I became accustomed with SQLObjects so at the moment this is what i have used. From a technical point of view im not sure which of the ORM's is best so this may change in future.

Well there you have it ... thats my software stack. I have other software packages installed but they are just tools ... OK more updates on the software build and the integration with the hardware later ...

Application, Software Stack ....

So what kind of application are we writing, hmmmm, dont want to reveal that just yet ;) however this is the software stack that we are using ...

• CentOS
• MySQL - (cluster)
• Cherrypy
• Python
• lighttpd
• SQLbjects

I'll discuss what these modules are and the interaction between each of these components of the software stack in a later post :) .... time to go to work .... :(

success on the internet ...

There have been alot of successes on the internet. More so during the dot com boom around the 2000 period, i think all that is over now as investors arnt as willing to invest in anything.com as they did back then. In the dot com boom any joe selling grass on the internet could be a millionaire.

That being said, i do think that there is somewhat of a boom, well not a boom but a willingness ot invest except that, its more controlled and not everything will make the creator a millionaire.

If you look at the recent web successes post 2000, most of them had some novelty or something that highy attacted the masses. Some of thes novelties that were present in some of the recent successes didnt seem like they had traction but eventually did gain traction and did become successful applications. With that being said, Lets look at some of the great successes of the internet ....

Google: Well probably the greatest success of all time and im glad. Couldnt have happened to a better pair of guys. They care about the world, provide the kind of working environment that any technology guru would feel welcomed in (instead of shunned and viewed as a *resource*). Google from the outset had alot of traction, their search was great and i think it was because of google and its searching ability that i was able to do alot of my assignements during uni ... back then nothing beat a google search when learning how to program :) ... so in saying that thanks to Sergey & Larry ... good job, keep creating the great technology and keeping the company creed alive ... 'dont be evil' (id shudder to think what would happen if google no longer took this creed seriously)

Youtube: A great success and something that i use very often, the idea is simple but is very useful. Its become the standard for online video delivery and has created many stars that have spun of making youtube videos. Different to google though, youtube's success was built by the population, it became what it is because of the users and the content they provided to drive popularity to youtube. I guess its like beez to honey, if you provide something everyone likes it will attract the masses.

Facebook: Another iconic internet website. Again another website that is created by the masses. Facebook provides the platform and people all around the world build the content. The ablity to share details, share experiences , share media and share information has created a website that is enourmously popular ... why, possibly because it allows us humans to endulge our normal human tendencies to share and let people know what we are doing in a way that is widley excepted ...

There are many other sites out there that are very popular that people may or may not know about, however still popular and sucessful none the less.

• Twitter


• Craigs List


• Dig


• Flicker


• mySpace


• Wikapedia

Simon - Left & Ryan - Right

I just want to also add a congratultions to some guys from my university where i did my undergrad - (university of sydney) who have succeeded in building a website/application that has launched them to stardom in the geek world !, specially in sydney. http://www.omnisio.com/ is their website and its a video mashup technology that allows editing of videos online by adding text bubbles, comments, combining powerpoint presentations along with many other features.

Google bought the technology and the guys are in the process of integrating the omnisio video mashup technology to youtube. Some of the features of their new technology have now been integrated into youtube and noticed them when i was uploading some of my own youtube videos.

I remeber these guys from my uni:

Ryan Junee: I had Ryan Junee as my tutor for operating systems and also for my initial blue (introduction to programming) class. Ryan was a great tutor and its good to hear of the successes of fellow university alumi's

Simon Ratner: Simon was in the same year as i was (if im not mistaken) and i remember seeing simon in our uni halls, seemed like he had always finished his assignments when all of us where doing late nights either in the labs or at home.... Also great to hear of the success simon has had ....

So ... how does this relate to us ... well i guess it just adds to the motivation and the driving force to succeed. I heard of Ryan and Simon's success half way during the development of our own little startup and even though Simon and Ryan are legends of the computer department of sydney uni, i feel that with enough hardwork our little startup can also be successful, how successful ... well i dont know, i guess time will tell.

Dare to dream ....

So for some reason i cant sleep, good thing is im in the mood to write, so lets write ! OK ... Dare to dream ... thats the title of this entry ... hmmm, its abit different then what i would usually write as i usually write about technical topics related to the implementation of the project. However I guess this entry is somewhat related cause its related to the inspiration behind the teams motivation to continue working ....

i wrote about the how all this started in 2 posts, the events that gave the ideas to begin this project and the initial driving force that prompted me to get off my but and 'JUST DO IT'. Starting is important but so is continous motivation .... I guess regardless of what i wrote earlier the reason why this project has continued to progress is every member of the team has dared to dream. If you think about it, who knows what will happen when we release this application for everyone to use, was this work we put in worth the effort, was it worth the time spent, is it a dud and did we just waste it all, if you think to much about it things start to play in your mind, but if you think of the end goal and what you want to achieve then you prevent any negative thoughts.

Regardless of what happens at the end, the journey that is currently taken is important. It builds the foundations and is also a learning experience. That being said most things in the life, regardless of the outcome, good or bad is a learning experience its really just our own interpretation.

So whats the dream ?!? The three guys working together on this project dream of running a successful and profitable business. Make some money and friends along the way, help some people and be free from the shackles of a 9-5. No one has told us we are going to be successful, but the dream keeps us going.

So looking back, below are some links to the youtube videos that have been posted to show the progress of the datacenter build since inception, since the days when the garage was cramped and full of rubbish, when we didnt have proper equipment ... to todays completion of the hardware build and half of the code we are deploying - (well the main components anyway) ... when looking back and watching the videos i think we can smile at the work we have done ...

OK, hope that wasnt too deep, i think im really gonna go to sleep now, below are links to the various you tube videos from Garage DC 1 - Garage DC 8 (more to follow later) nite ....

• GDC1 - http://www.youtube.com/watch?v=jZQiIIm_sdo
• GDC2 - http://www.youtube.com/watch?v=V-aAqg0GZ3w
• GDC3 - http://www.youtube.com/watch?v=ruCU9GAU5-k
• GDC4 - http://www.youtube.com/watch?v=-6CrRS7lDUc
• GDC5 - http://www.youtube.com/watch?v=fjldDs89mAA
• GDC6 - http://www.youtube.com/watch?v=dtXTRFQzttY
• GDC7 - http://www.youtube.com/watch?v=f5SJeTMfCbo
• GDC8 - http://www.youtube.com/watch?v=tUXpRdilryM

... nighters .... time for some ... zzzZZZzzZzz

Introducing the Garage Datacenter Artist ...

I would like to extend a warm welcome to garage datacenter artist ..... Garage datacenter artist contributes many different skill sets into the team that marry up very well with the current skill sets present ... hmmm, before we talk about datacenter artist lets look at the current team and the skill sets ...

• Garage CEO - (ME) : Garage CEO brings to the table, the idea's, the knowledge, the architecture skills and a broad scope of techical know how, extending, development skills, networking skills, linux/unix skills and a good understanding of technologies.



• Datacenter Hitman: Datacenter hit man is the Fix-It man. Give datacenter hitman any peice of hardware, damaged, not working, configuration errors and the Hitman will shoot the problem down and find a resolution. Datacenter Hitman has also extended his skills to great deal making and the ability to get things for dirt cheap prices. Datacenter hitman has been through the think and thin of the build and without the help of datacenter hitman much of this would not be possible.



• Datacenter Artist: Datacenter Artist is a new addition to the team. Datacenter artist brings to the table the design skills and the design know how to make a bland looking website that character and life. The garage datacenter team is relying on Datacenter artist to create mad designs which will enhance the usability of all the user interfaces. Datacenter artist also has good development skills - (as we went to uni together for a while) and will be able to help with the development and integration of the project .... i look forward to working with datacenter artist and looking at the designs that datacenter artist comes up with .....

So how does Datacenter Artist skills match up .... well currently the garage datacenter team is missing a design guru but with the addition of datacenter artist skills this is now no longer an issue. Also as a result of datacenter's ability to learn and understand code, itwill also help with the speed of development. Datacenter Artist also lives 5 minutes away so meeting up to finish some work is an easy task ... so again the garage datacenter team extends its welcome .... so if i was japanese i would say IRASHIMASEN .... - (welcome)

So there you have it, there are now currently three members. There have been guest appearances by various individuals who will get a mention later in latter sections of the record of events for the garage datacenter team ....

Thats about it for now, its getting late so time to hit the sack ... just about the quick picture above, its to show a small gathering for the datacenter team. To celebrate datacenter artist and Garage CEO's birthdays (AKA my birthday) we had a bit of a bbq at the datacenter, it actually wasnt t be because Dad had some things to do but he came home early so we just unpacked the meat and ate ! ... picture is of garage CEO and Dad of Garage CEO enjoying a satay, also present there was Datacenter Hitman ....

OK until next time :)

The Recap of the Garage Datacenter Build ....

Pictures are worth a thousand words arnt they, i have typed alot of text and provided alot of descriptions in my previous posts, but nothing beats a good old image & the odd video ;). So below are the images of the datacenter build over the last few months, starting from the days prior to the garage being available to the now currently completed equipment setup.

Its been quite a long road, with lots of challenges but we press on and continue to develop the system, the software and the documentation. I just want to say that im proud of the work that the guys have put in and its great to see what a bunch of guys can do when they work together.

Its a funny thing really, in the corporate world you have 10's of people on a team, dedicated for eight hours a day working on a particular task. People put on suites, and ties and talk *professional* but the funny thing is, dressed in shorts, ripped singlets a will to succeed, good team work and communication, we have done more over the last 8 months then what 10's of people do in the *corporate* world .... go figure. Even now i think we have achieved alot but for the garage datacenter team the true test of success is a system that provides a service that the general public can find useful and are willing to use ....

The road is still long but i think we are on the right path :)

So ok, without further a due, below ... the images ...

	There you have it, 8 months of hard work, configuration, setup and all our servers are up and configured. The database servers, application servers, network elements have been configured and tested. Tweaking is required to get best performance.
	The nortel switches have arrive and are waiting to be installed ...
	Another view of the switches ...
	Arrival of the second rack and our ML570 compaq proliant storage node ...
	Arrival of the first rack ...
	Racking of the first set of servers in the first rack ...
	Moving the two racks out whilst cleaning the garage ...
	Servers and workstations prior to buying the compaq racks...
	Same as above ...
	Purchase of the first set of dell machines to create the beowulf cluster ... This image shows the beowulf machines stacked up in my room. Machines stacked up whilst the garage is being prepared.
	Cleaing up the garage by sorting out all the old pc's. Sourcing materials as spares...
	Mum going through all the items in the garage to see if there is anything she needs to bring upstairs ...
	Taking out all the garage from the garage, the most painful part of the garage datacenter build ...

Procuring hardware the poor mans way .....

As a project manager in my 9-5 day job i see the different ways hardware is procured.

I see how hardware is procured when a company has a *preffered* service provider and uses this vendor for all hardware purchases regardless of the project and i also see how hardware is procured when the decision is made for selecting the *best* provider to supply hardware for a particular project.

Outsourcing was the way to go some years ago and possibly even now it is still the preffered method. It must be understood however that even outsoucing the management of infrastructure isnt as simple as saying to a hardware provider, here is my hardware go and support it !

There are many things that must be supplied to the vendor and there needs to be a certain amount of collaboration to succed in an out sourced model. When selecting the outsourced model, without correct management the kind of service you can end up with is a under par service, lots of finger pointing and millions of dollars worth of wasted money.

OK OK so what has this got to do with the garage datacenter you may ask ... Im getting to my point now .... :) so sorry for that little bit of a tangent ...

So here at the garage datacenter we procured hardware by looked at the kind of hardware we needed, the different channels we could get it through and the best possible savings we could achieve. We procured hardware and also bought backups incase of failure, this is mainly cause we are poor individuals and dont have millions or even hundreds of thousands of dollars to waste...

However, even though we saved alot of money on hardware, our datacenter has great equipment capable of running the system comfortably, all it takes is to purchase the right equipment, understand how to configure that equipment and maximize its throughput, takes bit of know how and abit of tweaking but the savings and the performances cant be beaten.

OK so lets take a look at how we procured hardware ... what did we do ?!?! Many of our hardware components where purchased as follows:

- Dell 2550's - Unverstiy of Technology Sydney throw away - Load balancers
- ML350's - Ebay - Application Servers
- ML370's - Ebay - Application Servers
- ML570's - Ebay - Storage Servers
- DL580's - Warehouse resellers - 4xDatabase Servers 2xFirewalls
- Racks - Ebay
- Switches - company seconds
- Dell p4 2ghz machines - Routers

Many of the challenges we had to overcome as a result of getting hardware at a really cheap, or almost free cost is as follows .... i guess time is money, but if you have some time and no money its better to spend time ;) ... So our issues that would waste our time:

- Damaged hardware - hard disks, missing back planes, memory modules, scsi cards
- Non standard hardware - Funny bolts and screws, different types of rails
- Non standard configurations that needed reverting - Funny pre-configured servers
- Getting components - Huge racks, heavy servers, lifting, mounting, racking
- Pickng up equipment

Myself and Datacenter Hitman had to pickup the racks ourselves and at times when it came to picking up some of the equipment we had to do it in the dead of night to avoid police detection. The reason for this was because some of the methods we used to bring back the equipment such as huge racks were less then what is regarded the *safe* way of doing things ;)

After all this heavy lifting though im glad to say our hardware is setup and completly configured from a hardware and systems software point of view. Along the way we also made very good friends with some of the people we dealt with and im sure in future these individeuals we met would be able to further assist us with out journey.

When i look back at the difficulties datacenter Hitman and I went through and what we have today , i sit back and think at whats more to come and the future ahead. Its been a great job and im proud (regardless of what happens) of the job data center hit man and I have done to get to this point.

For the next entry of this blog, i will add some comments of the additional team members who have joined our journey over the last few months ... this should complete the catchup posts and i should be able to post then from a day to day basis .....