Posted by Kromey at 3:00am Jul 21 '08So some of y'all ([private], I know you will!) may recall a while back I was talking about my future fileserver that would be teh awesome in all possible ways.
Well, the future is now!
4x 1TB SATA hard drives
1 GB RAM (temporary; will upgrade at least to 4 when $$$ permits)
Athlon X2 64 4200+
Dual on-board gigabit NICs
Sure, there's more specs, but this is all that really matters for a file server.
Although, it's more than merely a file server. I'll break it down in tasks, in roughly chronological order:
Task 1: Install Linux
Ubuntu 8.04.1 installed to a 10GB partition on one of the drives (I'll likely shrink that down later, I just didn't know how much I needed; so far I'm using just over 500MB). The first attempt failed (corrupted file errors), second attempt worked flawlessly.
Status: Complete
Task 2: Team NICs
Using the Linux bonding driver, my two gigabit NICs are now teamed together; this effectively gives my file server a 2 gigabit connection to my network and significantly reduces latency. There were some hassles getting this to work with DHCP, but I got it.
Status: Complete
Task 3: Configure DNS
It occurred to me that on my network there's not all that much of a heavy load for a file server, so this box can easily pull double-duty and also act as my own DNS server. This grants two main benefits: a) I can configure DNS entries for my home network, and b) I can bypass my ISP's annoying habit of hijacking bad DNS queries (i.e. non-existent domain responses) for their own ad-supported search site. I'm particularly proud of the one line of configuration that assigned 200 DNS names to my DHCP lease space!
Status: 90% complete (still have a few DNS entries to add)
Task 4: Configure LVM
4TB file server, but only 2TB (actually, less than that) will be used for files. 2 of the drives (less the system partition) will be partitioned together to form the main file storage; the other 2 will be partitioned together and serve to store versioned backups (more on that later). This will give me the redundancy of RAID, but also the flexibility to retrieve old backups of deleted or changed files if need be.
Status: Not yet started
Task 5: Configure backups
This will be an rsync-based utility that will take daily, weekly, and monthly snapshots of my files. It uses hard links for unmodified files, thereby dramatically reducing disk space needs, but breaks the links and copies over modified files. This means I'll be able to go to any single snapshot and see a complete picture of everything that existed then, but I won't be storing a gazillion copies of every single file to do it!
Status: Not yet started
Task 6: Install Subversion
This I'll use for versioning documents, giving me a much finer-grained ability to retrieve old versions and/or revert old changes. While not all that effective with binary files (music, pictures, movies), this will work great for documents (resumes, poems and stories, reports, etc.), provided they don't use Microsoft's binary-happy Office formats (ODF FTW!!). This repository will live among the main file server portion and will be backed up, but it will not be versioned (as it does that for itself already!).
Status: Not yet started
Task 7: ????
????????????????????????????????????
Status: Not yet started
Task 8: Profit!
Turn the results of Task 7 into cold, hard cash!
Status: Not yet started
The office right now looks like a veritable "Geek Zone": There's Mountain Dew cans and candy wrappers strewn about, 2-day old Chinese take-out behind me, two running computers, cables strewn haphazardly and dangerously across the only walk-way, which happens to exist between the file server and the decade-old CRT monitor and keyboard that provide me access to the server while I'm reconfiguring the network.
My girlfriend keeps asking me if I'm almost done yet, and I keep telling her that this is a multi-day, if not multi-week project to get fully set-up and running the way I want it to. And once this project is done, I have another old computer that I want to re-install Linux on and get back into writing code for miscellaneous little projects; I think my next one will be a simple IM client/server written in Python (because I'm learning Python right now). It'd also be great to start playing with developing and training AI again...
Well, the future is now!
4x 1TB SATA hard drives
1 GB RAM (temporary; will upgrade at least to 4 when $$$ permits)
Athlon X2 64 4200+
Dual on-board gigabit NICs
Sure, there's more specs, but this is all that really matters for a file server.
Although, it's more than merely a file server. I'll break it down in tasks, in roughly chronological order:
Task 1: Install Linux
Ubuntu 8.04.1 installed to a 10GB partition on one of the drives (I'll likely shrink that down later, I just didn't know how much I needed; so far I'm using just over 500MB). The first attempt failed (corrupted file errors), second attempt worked flawlessly.
Status: Complete
Task 2: Team NICs
Using the Linux bonding driver, my two gigabit NICs are now teamed together; this effectively gives my file server a 2 gigabit connection to my network and significantly reduces latency. There were some hassles getting this to work with DHCP, but I got it.
Status: Complete
Task 3: Configure DNS
It occurred to me that on my network there's not all that much of a heavy load for a file server, so this box can easily pull double-duty and also act as my own DNS server. This grants two main benefits: a) I can configure DNS entries for my home network, and b) I can bypass my ISP's annoying habit of hijacking bad DNS queries (i.e. non-existent domain responses) for their own ad-supported search site. I'm particularly proud of the one line of configuration that assigned 200 DNS names to my DHCP lease space!
Status: 90% complete (still have a few DNS entries to add)
Task 4: Configure LVM
4TB file server, but only 2TB (actually, less than that) will be used for files. 2 of the drives (less the system partition) will be partitioned together to form the main file storage; the other 2 will be partitioned together and serve to store versioned backups (more on that later). This will give me the redundancy of RAID, but also the flexibility to retrieve old backups of deleted or changed files if need be.
Status: Not yet started
Task 5: Configure backups
This will be an rsync-based utility that will take daily, weekly, and monthly snapshots of my files. It uses hard links for unmodified files, thereby dramatically reducing disk space needs, but breaks the links and copies over modified files. This means I'll be able to go to any single snapshot and see a complete picture of everything that existed then, but I won't be storing a gazillion copies of every single file to do it!
Status: Not yet started
Task 6: Install Subversion
This I'll use for versioning documents, giving me a much finer-grained ability to retrieve old versions and/or revert old changes. While not all that effective with binary files (music, pictures, movies), this will work great for documents (resumes, poems and stories, reports, etc.), provided they don't use Microsoft's binary-happy Office formats (ODF FTW!!). This repository will live among the main file server portion and will be backed up, but it will not be versioned (as it does that for itself already!).
Status: Not yet started
Task 7: ????
????????????????????????????????????
Status: Not yet started
Task 8: Profit!
Turn the results of Task 7 into cold, hard cash!
Status: Not yet started
The office right now looks like a veritable "Geek Zone": There's Mountain Dew cans and candy wrappers strewn about, 2-day old Chinese take-out behind me, two running computers, cables strewn haphazardly and dangerously across the only walk-way, which happens to exist between the file server and the decade-old CRT monitor and keyboard that provide me access to the server while I'm reconfiguring the network.
My girlfriend keeps asking me if I'm almost done yet, and I keep telling her that this is a multi-day, if not multi-week project to get fully set-up and running the way I want it to. And once this project is done, I have another old computer that I want to re-install Linux on and get back into writing code for miscellaneous little projects; I think my next one will be a simple IM client/server written in Python (because I'm learning Python right now). It'd also be great to start playing with developing and training AI again...
I'm so proud of myself! - Kromey