Sunday, December 07, 2008

Mac OS X 10.5.2 Leopard on a Dell Inspiron 531s

I've been for quite some time on seeing what it takes to get Mac OS X (Leopard) running on Intel PC hardware. For the last couple of weeks, I've done nothing but research and being a troll on forums and happened to stumble across a forum post about troubles with Mac OS X and a Dell Inspiron 530/531s. Cool thing is I actually have a Inspiron 531s, so the quest begins to make it happen

On several posts around the internet (most notably the insanelymac forums), there are claims that the 531s does/doesn't work, etc.

I'm here to tell you it does work and I was successful in getting Mac OS X 10.5.2 Leopard Leo4Allv3 version installed. It isn't fabulous by any stretch of the imagination just yet, but I'll explain below.

Just so it's not confusing or misleading, here are *my* hardware spec's on the Dell Inspiron 531s:

* AMD Sempron 3600+ (2.0Ghz, 256K cache)

* NVidia Geforce 6150SE nForce 430 (on-board, using 256MB physical RAM for video memory)

* 160GB SATA Hdd

* 1GB DDR2 SDRAM @ 667Mhz

* 48x CDRW/DVD drive


Below are the Mac OS X custom install settings for specific kexts I needed to enable to get it to work:

* Kernel-9.2.0
* Third Party Drivers
--> Chipset Drivers > NforceTest
--> Graphics Drivers > NVidia Driver > NVinject Choices > NVinject Version 0.1.3 > NVinject 256MB
* ACPIPlatform
--> Old-ACPIPlatform

Installation Directions (really brief):

1) I followed these directions almost verbatim, Click here to see them


2) When you get to the point in the Post-Install firstboot boot setup stuff where it asks "Do You Already Own a Mac?" when I would select the option "Do not transfer my information now" and click 'Continue' it would freeze temporarily and route me all the way back to the "Before You Begin" window. To fix that, I followed these directions:

a) Booted into single user mode ( -s at the boot> prompt)

b) fsck'd the filesystem and mounted the filesystem read/write per directions when in single user mode

c) Followed the commands below to fake out the firstboot crap and get into the OS (per the wiki link above):

# /sbin/fsck -fy
# /sbin/mount -uw /
# touch /var/db/.AppleSetupDone
# passwd root


On an operating system standpoint:

What Works:
* Well I can boot into Mac OS 10.5.2, yippie!
* NVidia 6150 at least works now and I get video, but it's really poor resolution
* CD-ROM

What Doesn't Work or isn't working (yet)
* On-board NVidia network card: I've found a post in insanelymac's form that I'm going to try this evening, but the Forcedeth/Forcedeth-V kexts from the install DVD did not discover my on-board NIC at all
* Have *no* idea how to get better results out of the on-board NVidia Geforce 6150. Right now, the only resolution I can do is 1024x768 and certainly no QE/CI. Any ideas?

What I haven't tested or am not able to test
* Audio: I think the video and NIC issue will consume most of my time right now. No sense adding more misery to the pile.
* Other network cards: I've read osx86 project's HCL and I do have gob's of hardware, problem is that the Inspiron 531s take low-profile NIC's (which I have none of) I could very well put in a standard PCI 10/100 NIC (Intel Pro100, 3Com 905, etc.) but the case side won't go back on and that looks tacky as hell.
* USB devices: Much like the audio dilema, I'm concentrating on the video and LAN, so this will come after.

So, I hope contributing this helps a lot of you out there. I'm horrible at maintaining posts and such, but I'm going to post this exact blurb on InsanelyMac (Mac OS X 10.5.2 + Leo4Allv3 + Dell Inspiron 531s Success) just in case I happen to forget to come back and post follow-up's to this.

I scored my Dell Inspiron 531s for $150 from a co-worker WITH 4 months left on the original 1-year warrenty, so I'm pretty pleased that I might be able to get Mac OS X going on it!

If any of you have any insight on some of the problems I'm experiencing, please feel free leave a comment!

Friday, October 31, 2008

Fd Linux development starting again... finally

It's been a *really* long time (approximately 7 years) since I've worked on my OSS project, Fd Linux and I'm finally starting to get my lazy ass in check and getting a bit more free time on my week[days,nights,ends] to get my development box set up and work on a new version of Fd Linux.

It's kind been a quirky hiatus from this due to getting deployed to Iraq back in 2003 to 2005 and just getting back into the swing of things, along with getting a house, watching my two girls grow up, and been extremely busy with work. Life kind of gets in the way like that.

Obviously over the years, the PC market has moved away from floppy disks (I honestly think floppy disk drives are *extra* commodities to add to PC's instead of defacto standards) and with solid state media dirt cheap right now and well capable of holding more than 1.44MB's worth of Linux goodness, it's a no brainer to start moving towards USB flash drives.

Right now, I'm working on a side project that will help get the base started for Fd Linux development, which is a home-brew NTP server, running on a PCB-5825 (AMD Geode 300Mhz, 128mb PC-133), booting off of 256MB compact-flash and using a Motorola Oncore UT+ GPS module with PPS as it's time source. There's much more detail to that, but a lot of the rootfs generation groundwork from way back to Fd Linux v1.0.0 will be used to concoct a good setup for this project.

Any rate, stay tuned. I'm pretty excited to get back into my groove.

Monday, August 04, 2008

Sparkfun = Cool, Customer Service = BAD

I'm a geek when I want to be and I really dig tinkering with EE, gadgets and hardware. SparkFun is one of those places you could easily spend days dreaming up oodles of projects to put together.

As much as I think they are cool, when a dude like me gets rubbed the wrong way one too many times from them, I tend to think otherwise. It all started back in Sept 2007; been looking for a breakout board for a GPS module I have and SparkFun had what I needed. I get the breakout board in the mail to find out there's design flaw on the board that won't even allow you to plug the certain GPS module I have into it. I contacted SparkFun, they owned up to the flaw and said they need to get rid of their current stock (which at the time was around 60-ish) so they can put the new stock up with the revision.

Well... after several e-mails, many lies and false hopes that the *next* batch of breakout boards were the ones I needed with this so-called 'revised' fix, on July 30, 2008 (almost 10 MONTHS!) I was notified that the new revision was finally in and placed an order.

Today, August 3, 2008, I get the long, awaited package. Do you think it was this breakout board for my GPS module? Nope. It was something complete different; something that wasn't even in the same fucking realm as to what I was expecting. It was like ordering a pizza and getting ape shit in box delivered instead.

So I most certainly called immediately and what's really *odd* about this is, it's like they were expecting my call. I was transferred to the gentleman that apparently packaged and/or put together my order and the first thing he said was, "Here's your 2-Day FedEx tracking number for your item you didn't get. We'll be enclosing a return shipping label for the wrong parts we sent you.", all within the first 20 seconds of the conversation. It's like they were expecting me to call because of one two reasons: 1) Human error, or 2) They *said* that their PCB revision was in but it really is not, and did not have their run finished, so they sent me bogus shit in return to stall while their production run finished. You be the judge on the conspiracy theory...

The point I'm trying to make here is: SparkFun has cool shit and I might buy from them again, but if they fuck up, don't expect them to win your heart and mind over.

Saturday, August 02, 2008

PAM max login limits in RedHat Enterprise 5

I found something surprisingly 'odd' with pam_limits.so library when setting up and base lining a few servers the other day at work.

For some reason you can not concurrently use both 'maxsyslogins' and 'maxlogins' at the same time in your /etc/security/limits.conf file to set max authenticated user/group logins per user or system. With my intentions of having a limits of anyone part of, lets say, the 'users' group, could not have more than 3 separate simultaneous logins into the box with a maximum hard limit of 15 total system logins just doesnt seem to work.

So if you're going to try something like this:



@users hard maxlogins 3
* hard maxsyslogins 15



...didn't work for me. I have noticed, also, if you have a particular user who is part of several groups (e.g. users and devel groups) and you have limits for both of those groups, once the first group limits is matched, it goes with that rule. But if you have any idea on how to get 'maxlogins' and 'maxsyslogins' to work together, I'd be happy to know.

Monday, June 16, 2008

Analyzing and cleaning out Linux/Unix filesystems that fill up

So I have this problem from time to time at work: bastid users or custom processes that either go rogue and start filling up filesystem space at an alarming rate, or crappy code that doesn't clean itself up.

Here's a couple of my favorite commands to help out with that:

`find` Command

If you aren't using `find`, you should. My favorite one-liners for stuff like this:

Find all files in a directory with a size between (for example) 1KB and 10MB that last changed 1 minute(s) ago:

[root@testbox]# find . -type f -size +1024k -and -size -10240k -cmin 1 -exec du -sb '{}' \;


Find all files in a directory with a size at least (for example) 75MB that last changed 1 minute(s) ago:

[root@testbox]# find . -type f -size +76800k -cmin 1 -exec du -sb '{}' \;


Find all files in a directory with a size between (for example) 1MB to 100MB, owned by 'frank' (UID: 501) that last changed 1 minute(s) ago:

[root@testbox]# find . -type f -size +1024k -and -size -102400k -uid 501 -cmin 1 -exec du -sb '{}' \;


...the execution of `du` in my commands will definitely add some overhead, so feel free to leave it off. It's a nice quick human readable presentation if you're in a hurry, but I wouldn't do it if you have a shitload of files that are going match your search.

`lsof` Command

Simply: list open files. This is your friend you can use in tandem with the `find` commands above to narrow down what process(es) are pillaging your filesystem.

What I do is take the output of a particular find command if I see growing files out there but cannot narrow down what process(es) it could be and just simply run:

[root@testbox]# lsof | grep file_name

...where obviously, file_name is some output you got from the `find` command(s).

Those are just some of many tricks to do to track down that type of ridiculousness that happens when things go bad late at night when you're on-call.

Tuesday, May 27, 2008

WEP pwn'age... what's next?

After all these years of known vulnerabilities with the technology, I'm still flippin' amazed the percentage of deployed, neighborhood 802.11b/g wireless access points that STILL have WEP encryption enabled. Not saying that WPA-TSIP-PSK is much better (well, if you've got a super, stellar dictionary/phrase list) but... what this oversight gives me is two things:

1) Entertainment for a very short period of time in 20 minute increments...
2) A very big 'network' to explore, play with and hide behind...

I can't give myself all the credit; `aircrack-ng` is a pretty slick tool.

However, just tonight, I've pwn'ed the last WEP-enabled access point on my block... well, all the ones I can reach without using my 20dbi gain omni antenna poking out my basement window ;-) ...now all that is left is burning my braincells watching television and reading slashdot.

Saturday, May 17, 2008

Cisco VPN client and Fedora Core 8

I recently did a kernel upgrade on my laptop and noticed my Cisco VPN software wasn't working due to needing to recompile the Cisco `vpnclient` code to build the needed modules again.

...and I discovered two things: what I forgot and what I didnt forget.

1) Forgot: how fucking ridiculously broken kernel code is in Fedora's distribution stream. OMFG. Can a person compile anything outside hello_world.c against anything released? (harsh and far from true, but I'm pissed)

2) Didn't Forget: how painful it was to get `vpnclient` going again on my laptop the first time with all the patching and 32/64-bit specific OS crap to take into consideration.

Want the steps? Here you go.

Cisco VPN client packages and patches

I'm running Fedora 8 64-bit on my laptop, and this are the packages that worked for me. Get 'em:



Unpack and patch source

Next, do the following, assuming you've put all this in the same directory or sandbox:



[testbox]$ tar -zxvf vpnclient-x86_64-4.8.01.0640-k9.tar.gz
[testbox]$ mv vpnclient vpnclient-x86_64-4.8.01.0640-k9
[testbox]$ cd vpnclient-x86_64-4.8.01.0640-k9
[testbox]$ patch -p1 < ../vpnclient-linux-2.6.24-final.diff
[testbox]$ patch -p1 < ../cisco_skbuff_offset.patch



...we'll also need to do a quick hack on the 'Makefile' to make `make` happy.



[testbox]$ cp Makefile Makefile.orig
[testbox]$ sed -i -r -e "s/^CFLAGS/EXTRA_CFLAGS/g" Makefile



Compile Cisco VPN source

Ok, now the part you've been waiting for: compile time. Do the following (noting that doing the install requires r00t level access, so I use `sudo` for my needs, do whatever suits yours):



[testbox]$ cd vpnclient-x86_64-4.8.01.0640-k9
[testbox]$ sudo ./vpn_install



...and then follow the on-screen instructions for installation path, etc. etc. etc.

Testing the Cisco VPN client

Assuming that the compilation of the kernel modules went successfully, now it's time to test out the Cisco VPN client. First, make sure you're actually connected on 'some' network that's going to allow you to get to your VPN. Second, make sure you copy your Cisco VPN profile out in /etc/CiscoSystemsVPNClient/Profiles so you can actually connect to your VPN.

After that, do the following:



[testbox]$ sudo /etc/init.d/vpnclient_init start
[testbox]$ sudo vpnclient connect [name_of_profile_you_copied]



...and that's pretty much about it in a quick and dirty way. Hopefully this helps someone. I wasn't going to even attempt to start hacking code to see what was really broken. I had google'd around and noticed that people had it working; the tough part was finding all the correct patches to go with certain code bases (e.g. 64-bit for my 64-bit OS I'm running).

Sunday, May 04, 2008

Marvel's 'Iron Man' a MUST SEE!

I'm not a huge comic book fan; I grew up on the edge of the era and although I do have my fair share of DC comics (Batman, Superman, etc.) I have to say I've always been intrigued with super-hero marvels from Marvel comics. X-men, Spiderman, and the Hulk are obviously the king names in super hero nostalgia.

Marvel had their push of comicbook-turn-theatrical run of movies so far, such as Ghostrider, X-men, Fantastic-4, etc. And I think the only one that I never was fond of was Ghostrider.

'Iron Man' theatrical debut was this Friday, May 2, 2007. I went Saturday night and I have to say it was one of the BEST MOVIES, comic book movie comparisons aside, that's I've seen in a LONG time. It had the right amount of humor, the actors/actresses were perfectly picked, the special effects were excellent and not overly done.

If you haven't see it, I recommend you go. And if you do, WAIT until the end credits have roled, there's a special 2 minute blurb at the end that obviously spells out the sequal to 'Iron Man': The Avengers!

Saturday, April 26, 2008

The re-birth of the subnotebook?

It's funny how things in our culture resurface themselves after decades of silence. From fashion to technology and anywhere in between, it's crazy the amount of odd things that are all the sudden "the thing to have" today.

Odd shit like kids wearing straight leg pants that were cool in the 80's, girls wearing their hair in a pony tail that drags off to the side of their shoulder, and most notably (of course I'm not going to bitch about fashion) the re-birth of the subnotebooks like the Asus Eee and Everex Cloudbook.

...so I ask myself: WTF happened to the Gateway Handbook? I happen to own one of these (and still do) and at the time, I used to run my own homebrew Linux distro on it (Fd Linux), in which I ripped out the 80mb 2.5" hdd and tossed in a IDE micro-controller-to-compact flash, which essentially, despite the shear lack of processing power that the Asus Eee and the Everex Cloudbook boast, isn't much different.

I'd definitely have to say, we are creatures of habit and repetition.

Sunday, April 20, 2008

USB Flash Drive Replacement Caps!

Just like anyone, it's a sad day when you finally misplace or break the cap removable cap from your USB flash drive. I, myself, become reluctant to use it any more in fear of getting the USB header bent or mangled in my bag I bring to work.

Well, fear no more, there's a company called RooKaps that makes rubber replacement USB flash drive caps (in assorted colors) AND lanyard-ifed caps, too!

At $5.77+S&H for four of them (non-colored and non-lanyard), it's not the cheapest deal, but it's much cheaper than say, purchasing a new USB male PCB header, cracking the shell on your USB drive (which won't go back together easily at all), and re-soldering the new header on.

Buy some, I am.

Sunday, April 13, 2008

Using newzbin with hellanzb and hellafox in Linux

I'm a huge newzbin fan! However, I could never find anything intuitive out there that was newzbin file friendly in the Linux world, as far as a .nzb file parser/reader and pull all that wonderful 'content' down from my ISP's Usenet (news) server.

I found a cool app out there called 'hellanzb'. It's a command line tool to process, queue, download or pull down .nzb files from great ease from newzbin's webstie (not to mention the automatic capability to unrar, unzip or unpack your downloads...mmm). Use it in conjunction with 'hellafox', and you've got a nice, streamlined ticket do download all the indexed Usenet 'content' a person would want.

I figured this was blog worthy, because there were some pretty shitty directions out there to get this going.

Prerequisites

Since I'm using Fedora 7 on my laptop, I had to make sure these packages were installed: python, python-twisted, unrar and par2cmdline. Again, this might not be 'everything' you need to be successful, but it's a good start for getting `hellanzb` and it's auto-unrar-ing and par2 checking features working.

If you don't have any of these installed, then just type this:

[testbox]$ yum install python python-twisted unrar par2cmdline -y


Download and Compile hellanzb

1) Download hellanzb tarball from http://www.hellanzb.com


[testbox]$ cd /home/user/sandbox
[testbox]$ wget http://www.hellanzb.com/distfiles/hellanzb-0.13.tar.gz


2) Unpack the tarball
[testbox]$ tar -zxvf hellanzb-0.13.tar.gz


3) Compile and install (as super-user)


[testbox]$ cd hellanzb-0.13
[testbox]$ sudo python setup.py install



Setting up and Configuring hellanzb.conf

The hellanzb python script (hellanzb.py) is defaultly installed in /usr/bin. As for the configuration file (hellanzb.conf), it's defaultly installed in /usr/etc. At install time, the file /usr/etc/hellanzb.conf.sample exists, so lets make a real, working copy of that so we can edit it:



[testbox]$ sudo cp /usr/etc/hellanzb.conf.sample /usr/etc/hellanzb.conf
[testbox]$ chmod 755 /usr/etc/hellanzb.conf



...now open the file with everyone's favorite text editor, let's edit hellanzb.conf and make some changes:

[testbox]$ sudo vi /usr/etc/hellanzb.conf


Here my example hellznzb.conf configuration file, with the most important settings highlighted in red:



# Log output to this file, set to None (no single quotes) for no logging
Hellanzb.LOG_FILE = '/var/tmp/hellanzb.log'

# Uncomment this line to log DEBUG messages to the specified file
Hellanzb.DEBUG_MODE = '/var/tmp/hellanzb-debug.log'

# Automatically roll over both log files when they reach LOG_FILE_MAX_BYTES
# size
Hellanzb.LOG_FILE_MAX_BYTES = 2048

# Save LOG_FILE_BACKUP_COUNT of those rolled over log files
Hellanzb.LOG_FILE_BACKUP_COUNT = 0

defineServer(id = 'servergrp',
hosts = [ 'your.news.server..hostname.net:119' ],
#hosts = [ 'news.changeme.com', 'morenews.changeme.com:8000' ],



# If your ISP allows you to log onto their news server with a login/password,
# uncomment and put those credentials in below
#username = 'usenetID',
#password = 'usenetPASS',

# If your ISP does not have credentials for you to access it's newserver, uncomment
# these lines below
username = None, # no auth
password = None,


connections = 4,
antiIdle = 4.5 * 60, # 4 minutes, 30 seconds, 0 to disable
#bindTo = '204.31.33.7', # connect FROM this ip address
#enabled = False, # disable this server
#skipGroupCmd = False, # skip sending nntp GROUP commands
#fillserver = 0, # defaults to 0 (a main server).
# fillservers must have values > 0
# (priority)
ssl = False
)

# Uncomment this line to limit all server connections to the specified KB/s
# bandwidth
#Hellanzb.MAX_RATE = 150 # limit to 150kB/s

# Important locations
Hellanzb.PREFIX_DIR = '/home/user/newzbin/'

# Where to put queued .nzb files
Hellanzb.QUEUE_DIR = Hellanzb.PREFIX_DIR + 'nzb/daemon.queue/'

# Where the fully processed archives go
Hellanzb.DEST_DIR = Hellanzb.PREFIX_DIR + 'completedownloads/'

# The .nzb currently being downloaded is stored here
Hellanzb.CURRENT_DIR = Hellanzb.PREFIX_DIR + 'nzb/daemon.current/'

# The archive currently being downloaded is stored here
Hellanzb.WORKING_DIR = Hellanzb.PREFIX_DIR + 'nzb/daemon.working/'

# Archives interrupted in the middle of downloading are stored here temporarily
Hellanzb.POSTPONED_DIR = Hellanzb.PREFIX_DIR + 'nzb/daemon.postponed/'

# Archives currently being processed. May contains archive directories, or
# symbolic links to archive directories
Hellanzb.PROCESSING_DIR = Hellanzb.PREFIX_DIR + 'nzb/daemon.processing/'

# Temp storage
Hellanzb.TEMP_DIR = Hellanzb.PREFIX_DIR + 'nzb/daemon.temp/'

# Filename to store hellanzb state in between CTRL-Cs. The state (includes the
# order of the queue, and smart par recovery information) is intermittently
# written out as XML to this file
Hellanzb.STATE_XML_FILE = Hellanzb.PREFIX_DIR + 'nzb/hellanzbState.xml'

# _Sub directory within the nzb archive dir_ to move processed files to
Hellanzb.PROCESSED_SUBDIR = 'processed'

# Remove the PROCESSED_SUBDIR if the archive was successfully post processed.
# Warning: The normal Hellanzb.LOG_FILE should be enabled with this option --
# for a record of what hellanzb deletes
Hellanzb.DELETE_PROCESSED = True

# Maximum amount of memory used to cache encoded Article data segments.
# hellanzb will write article data to disk when this cache is exceeded
# Available settings:
# -1: Unlimited size
# 0: Disable cache (only cache to disk)
# >0: Limit cache to this size, in bytes, KB, MB, e.g.:
# 1024 '1024KB' '100MB' '1GB'
#Hellanzb.CACHE_LIMIT = 0


# Save archives into a sub directory of DEST_DIR named after their newzbin.com
# category (when queued using the enqueuenewzbin XMLRPC call); e.g. Apps,
# Movies, Music
Hellanzb.CATEGORIZE_DEST = True

# Disable SMART_PAR (download all PAR files)
Hellanzb.SMART_PAR = False
#Hellanzb.SMART_PAR = True

# Supply a path to the (un)rar command
# In case in cannot find `unrar` in your PATH environment
#Hellanzb.UNRAR_CMD = None

# Supply a path to the par2 command
# In case in cannot find `par2` in your PATH environment
#Hellanzb.PAR2_CMD = None

# Skip unraring during post processing
Hellanzb.SKIP_UNRAR = False

# Supply a path to the optional macbinconv command (for converting MacBinary
# files)
#Hellanzb.MACBINCONV_CMD = None

# hellanzb inherits the umask from the current user's environment (unless it's
# running in daemon mode). The umask can be forced with this option
Hellanzb.UMASK = 0022

# Supported music types (case insensitive) and optionally their decompression
# executables
# and the file type that executable will decompress to (case insensitive). The
# exes must be in the PATH.
#
# will be replaced with the name of music file
# optional is with the specified extension
#
# None means these files don't need to be decompressed
defineMusicType('wav', None, None)
defineMusicType('mp3', None, None)
#defineMusicType('ape', 'mac -d', 'wav')
#defineMusicType('flac', 'flac -d -- ', 'wav')
#defineMusicType('shn', 'shorten -x < > ', 'wav')

# Max files we should decompress at the same time
Hellanzb.MAX_DECOMPRESSION_THREADS = 2

# Enable Mac OS X Growl notifications
Hellanzb.GROWL_NOTIFY = False

# The growl notification server, in the format 'hostname'
Hellanzb.GROWL_SERVER = 'IP'

# The growl password
Hellanzb.GROWL_PASSWORD = 'password'


# Enable libNotify Daemon notifications
Hellanzb.LIBNOTIFY_NOTIFY = False


# Disable ANSI color codes in the main screen (preserves the in place scroller)
#Hellanzb.DISABLE_COLORS = False

# Disable ALL ANSI color codes in the main screen (for terminals that don't
# support ANY ANSI codes
#Hellanzb.DISABLE_ANSI = False


# Hostname for the XMLRPC client to connect to. By default, localhost
Hellanzb.XMLRPC_SERVER = 'roboto'

# Port number the XML RPC server will listen on, and the client will connect to.
# Set to 'None' (without the quotes!) for no XML RPC server
Hellanzb.XMLRPC_PORT = 8760

# Password for the XML RPC server. You might probably never use this, but the
# command line XML RPC calls do -- it should definitely be changed from its
# default value. The XML RPC username is hardcoded as 'hellanzb' -- E.g. URL:
# http://hellanzb:changeme@localhost:8760
Hellanzb.XMLRPC_PASSWORD = 'newzbin'

# Username/Password to http://www.newzbin.com for automatic NZB downloading
Hellanzb.NEWZBIN_USERNAME = 'newzbinacct'
Hellanzb.NEWZBIN_PASSWORD = 'password123abc'

# If any of the following file types are missing from the archive and cannot be
# repaired, continue processing because they're unimportant (case insensitive)
Hellanzb.NOT_REQUIRED_FILE_TYPES = [ 'log', 'm3u', 'nfo', 'nzb', 'sfv', 'txt' ]

# Don't get rid of (move into the PROCESSED dir) the following file types when
# finished post processing (case insensitive)
#Hellanzb.KEEP_FILE_TYPES = [ 'log', 'm3u', 'nfo', 'nzb', 'sfv', 'txt' ]
Hellanzb.KEEP_FILE_TYPES = [ 'nfo', 'txt' ]


# List of alternative file extensions matched as NZB files in the QUEUE_DIR.
# The 'nzb' file extension is always matched
#Hellanzb.OTHER_NZB_FILE_TYPES = [ 'xml' ]

# Support extracting NZBs from ZIP files with this suffix (case insensitive) in
# QUEUE_DIR. Defaults to '.nzb.zip'. Set to False to disable.
#Hellanzb.NZB_ZIPS = '.nzb.zip'

# Support extracting NZBs from GZIP files with this suffix (case insensitive)
# in QUEUE_DIR. Defaults to '.nzb.gz'. Set to False to disable.
#Hellanzb.NZB_GZIPS = '.nzb.gz'

# Delay enqueueing new, recently modified NZB files added to the QUEUE_DIR until
# this many seconds have passed since the NZB's last modification time (defaults
# to 10 seconds)
#Hellanzb.NZBQUEUE_MDELAY = 10

# Optional external handler script. hellanzb will run this script after post
# processing an archive, with the following arguments:
#
# handler_script type archiveName destDir elapsedTime parMessage
#
# type: post processing result, either 'SUCCESS' or 'ERROR'
# archiveName: name of the archive, e.g. 'Usenet_Post5'
# destDir: where the archive ended up, e.g. '/ext2/usenet/Usenet_Post5'
# elapsedTime: a pretty string showing how long post processing took, e.g.
# '10m 37s'
# parMessage: optional post processing message. e.g. '(No Pars)'
Hellanzb.EXTERNAL_HANDLER_SCRIPT = '/home/user/newzbin/post_results_nzb.pl'



...most of that configuration file is self-explainatory. If you're in a hurry to get it up and go, definitely pay attention to the hosts, username, and password lines for your newsserver in the defineServer () block and if you decide NOT to use hellafox, put in your newzbin account credentials in the NEWZBIN_USERNAME and NEWZBIN_PASSWORD or you'll pretty much have no luck downloading anything or browsing newzbin indexes via command line.

One of the coolest features (I think) is the last line of the configuration file, which EXTERNAL_HANDLER_SCRIPT. This is *WAY* cool in my opinion! It basically gives you flexibility to have hellanzb put download posting result parameters into your custom script for notification when your downloads are complete!

...of course, I just had to try this out and it's 'hella' wicked (what a pun!) I whipped up a very crude perl script that will pop me up a simple perl/Tk GUI with the download results, so I'll get instant notification when my downloads are complete:



#!/usr/bin/perl
#
#
# Creation Date: 4/13/2008
#
# Script Name: post_results_nzb.pl
#
# Author: Adam Dosch
#
# Description: A post notification script written perl/Tk to notify me when my Usenet downloads are finished via hellanzb using newzbin indexing files (.nzb files)
#
# Licensing: Free to use and distribute, just give credit where it's due (e.g. sling your OWN code if you're gonna rip off mine)
#
#################################################################################################################
# START OF SCRIPT - DO NOT EDIT BELOW UNLESS YOU KNOW WHAT YOU ARE DOING
#################################################################################################################

#
# Perl Library Definitions
use Tk;

#
# Subroutines

sub quit_script
{
exit 0;
}

#
# Processing Input parameters from HellaNZB
#
# Here's the input we are expect, straight from hellanzb.conf:
#
# handler_script type archiveName destDir elapsedTime parMessage
#
# ARGV[0] -> type: post processing result, either 'SUCCESS' or 'ERROR'
# ARGV[1] -> $archiveName: name of the archive, e.g. 'Usenet_Post5'
# ARGV[2] -> $destdir: where the archive ended up, e.g. '/ext2/usenet/Usenet_Post5'
# ARGV[3] -> elapsedtime: a pretty string showing how long post processing took, e.g.
# '10m 37s'
# ARGV[4] -> parmessage: optional post processing message. e.g. '(No Pars)'
#

# Because we expect 6 parameters at all times, I just did it the crude way and
# use @ARGV without validating anything other than if we have parameters or not.
# What a programmer, eh?


if (@ARGV) {

#
# Main Window that shows post results

my $mw = new MainWindow;
my $labelheading = $mw -> Label(-text=>"NewzBin Download - $ARGV[0]") -> pack();
my $labelspacer1 = $mw -> Label(-text=>"") -> pack();
my $labelname = $mw -> Label(-text=>"Name: $ARGV[1]") -> pack();
my $labeldir = $mw -> Label(-text=>"Directory: $ARGV[2]") -> pack();
my $labeltime = $mw -> Label(-text=>"Elasped time to process: $ARGV[3]") -> pack();
my $labelpars = $mw -> Label(-text=>"Any PARS?: $ARGV[4]") -> pack();

my $button = $mw -> Button(-text => "Quit", -command => sub { exit 0} ) -> pack();
MainLoop;

} else {
# No paramters or something crazy happened!
die "No parameters passed!" && exit 1;
}



...once you've got everything configured the way you like, now it's time to use hellanzb!

Start hellanzb

To start `hellanzb`, it's quite simple. It's installed in /usr/bin/ and it's called 'hellanzb.py'. Just as long as you made your hellanzb.conf file permissions world readable (755), then you can just start up the hellanzb app either in standalone/debug or in daemon mode as yourself (non-root).

To start it up as a standalone/debug mode, which will tie up a terminal, but put all the server output to SDTOUT, do:
[testbox]$ /usr/bin/hellanzb.py


You can also daemon-ize this (which I recommend), by doing:
[testbox]$ /usr/bin/hellanzb.py -D


...if you enabled any of the hellanzb logging in your configuration file and you want see or troubleshoot something that's going on, just `tail` the log files hellanzb.log and hellanzb-debug.log which were defaultly located in /var/tmp.

Using hellanzb

I'm not going to dive into the semantics of using the command-line operations of `hellanzb` because this blog post would be HUGE and I'm not going to go into the ways I power-use it. If you want a list of all the command-line syntax and options for hellanzb, type:

[testbox]$ /usr/bin/hellanzb.py -h


However, the easiest way if you don't want to integrate hellafox into the mix (which is the hellanzb FireFox extension), is to:

1) Log onto www.newzbin.com with your newzbin credentials

2) Find the newzbin file you want to download, download it and save it to the Hellanzb.QUEUE_DIR location that you defined in your configuration file and hellanzb will automatically start processing and downloading your stuff.

3) Then if you happen to use the notification script or if you're tail'ing the log files to see when your download is done, go out to your Hellanzb.DEST_DIR and there it will be!

...so, that's the quick and dirty version to getting hellanzb working for you under Linux! If you want to configure hellafox, the FireFox browser extension, which will allow you to surf the newzbin.com website and send all your requests to download with a few mouse-clicks, continue on!

Downloading and Installing hellafox

`hellafox` was kind of tricky to get going, but I essentially went to FireFox's add-ons site and installed the add-on and restarted the browser. I do highly recommend reading hellafox's site for how to do this, so I don't have to completely reinvent the wheel.

Configuring hellafox

To configure hellafox will rely on the following configuration options you set up in your hellanzb.conf file above. Make sure you have the following information available from your configuration file, as you will need it to set up the hellafox add-on:


Hellanzb.XMLRPC_SERVER #Hostname of the machine running hellanzb, according to hellafox setup
Hellanzb.XMLRPC_PORT #The port upon which hellanzb is listening, according to hellafox setup
Hellanzb.XMLRPC_PASSWORD #The password used to connect to hellanzb, according to hellafox setup


To configure hellafox:

1) Launch Firefox

2) Goto Tools > Add-ons

3) Click the "Preferences" button under 'Hellafox firefox extension' add-on and put in the following information you gathered above (RPC server hostname, listening port and password)

Once that is done, simply do this:

1) Log onto www.newzbin.com with your newzbin credentials

2) Scour the site for whatever you're looking for, but like the hellafox website directions say, just right-click on the post link you want to download and select "Downlad this NZB"

...and away you go! Have fun and good luck!

Thursday, April 03, 2008

Corruption Case: The Wii Controversy and Wal-mart

So I decided to join the bandwagon and broke down and purchased a Wii. I do have a to say I'm not sure why I waited so long! It's pretty damn fun, that's for sure and well worth the $318.

One thing I'm not proud of is the place I purchased my Wii from. If you know me personally, you know that I despise Wal-mart with a passion. What was disgustingly wrong about the whole Wii purchasing experience is that despite my wife picking the right day by luck to show up to ask for Wii's, the minute the Wii's arrived off the UPS truck into the electronics department, Wii's in groups of two were instantly scooped up by the bottom feeder Wal-mart employees with the insider tip that Wii's were coming in that day.

In a matter of minutes, 4 of the 11 Wii's were instantly gone and would never be seen by a consumer eye. Two more Wii's went to the local Pepsi delivery man who had befriended a few Wal-mart employees.

Case in point: Wal-mart is a sess pool of shit. Not just for the fact that consumers don't get the fair shake in because of shady employed Wal-martians when it comes to Wii's, but because of their tight life sucking reins on the consumer market.

Monday, March 31, 2008

Poor Man's NAS: RAID Fun and disk optimization with Fedora Core, Part 1

I'm sure any half way tech-savvy idiot has noticed that the disk-storage-to-cost ratio has considerably pointed in the favor of the home computer user in recent years, in that you can get a boat load of disk space for a very small amount of money. Well, I've finally accumulated enough spare parts to spark an interest in finally putting together a home-brew NAS on my home network for all my junk (mp3's, movies, on-the-fly backup network backup solution, kickstart environment, etc.).

(This is part 1 of a couple part article. I'm only sectioning it off because I can get very long winded at times.)

Here's what I used to put this together:

* PIII-866 512MB RAM (Got mine for free at the local school)
* 40gb IDE boot harddrive
* Any Linux OS (Free, I used Fedora Core 7)
* 2 - Promise ATA-100 PCI EIDE Controllers
* 3 - Western Digital WD1200 120GB IDE harddrives
* Intel Express E100 10/100 PCI NIC

I had all this junk lying around my house, so it really cost me nothing. I, myself, do not need some 2TB NAS on my network, so again, this is the thrifty approach here. Most of this stuff I've gotten for free over the years anyway, so if you're constantly in the used computer hardware market like me, you'll have no problems putting something like this together and not spending much of anything do it. It's pretty obvious I don't throw much stuff away either...

Server Assembly

This is more of a cautionary-slash-design note than anything. You might be asking why I picked to use 2 ATA controllers instead of just one (e.g. I have 3 drives, and with a primary and secondary controller on one card, that allows 4 total devices)? The reason is performance and reliability. If you have two IDE disks on the same controller chain, you'll see a performance hit. The reliability factor is that if you happen to have a bad disk or bad controller, you're going to loose two drives instead of one because of a failure. Since I'm planning on using a RAID-5 configuration without a hot spare, putting the individual drives each on an individual controller made the most sense to me. However, I also plan to contrast the differences in using RAID-0, which it's only advantage for me would be speed verses redundancy to the point with RAID-5 that I'd have time to go get another drive and replace it and n-1 concept of the parity drive and eating 120gb in the pants.

Harddrive Discovery and Optimization

After the box was assembled and the Linux OS is installed, first thing to do is to gather the device names of all the disks you are going to use in your RAID configuration. I'm looking for my 3 Western Digital drives that live on my Promise ATA controllers. Best way to do this is simply do some fancy grepping on the `dmesg` output and then look for your disks:



[testbox]& dmesg | grep ^hd[a-z]:
hde: WDC WD1200BB-00CAA2, ATA DISK drive
hdg: WDC WD1200BB-00CAA1, ATA DISK drive
hdi: WDC WD1200BB-00DWA0, ATA DISK drive

hda: Maxtor 32305U3, ATA DISK drive
hdc: IDE DVD-ROM 16X, ATAPI CD/DVD-ROM drive
hde: max request size: 128KiB
hde: 234493056 sectors (120060 MB) w/2048KiB Cache, CHS=16383/255/63, UDMA(100)
hde: cache flushes supported
hdg: max request size: 128KiB
hdg: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
hdg: cache flushes not supported
hdi: max request size: 128KiB
hdi: 234441648 sectors (120034 MB) w/2048KiB Cache, CHS=16383/255/63, UDMA(100)
hdi: cache flushes supported
hda: max request size: 128KiB
hda: 45023328 sectors (23051 MB) w/512KiB Cache, CHS=44666/16/63, UDMA(100)
hda: cache flushes not supported
hdc: ATAPI 44X DVD-ROM drive, 512kB Cache, UDMA(33)



The disks I'm looking for (which are in bold) are the Western Digital WD1200's I want use for my RAID-5 configuration.

Next, you'll want to check to see what type of disk performance you're going to be looking at getting. To do this, you'll definitely want to make sure you have the `hdparm` package installed. In Fedora, to install this package if you don't have it, do the following:



[testbox]& yum install hdparm -y



`hdparm` is a pretty powerful hard disk ioctl tool, that gives you direct access into tweaking disk features, along with a very rough, but somewhat accurate read/buffer testing.

Right away when I first booted up the box with the Western Digital drives, I ran a buffered read and cache test on the drives, and noticed on a few of them that I was getting some really poor-ass performance:



[testbox]& hdparm -Tt /dev/hde

/dev/hde:
Timing cached reads: 992 MB in 2.00 seconds = 494.97 MB/sec
Timing buffered disk reads: 12 MB in 3.00 seconds = 4.00 MB/sec



...which is terrible theoretically considering ATA-5 drives on an ATA-100 controller should get, at best, roughly 50-60mb/sec buffered disk reads from my experience. So how'd I solve this? With one acronym: DMA (e.g. Direct Memory Access). DMA is essentially allowing your EIDE devices (such as a harddrive or CDROM) to bypass your microcontroller and have direct transfer to your computer's memory.

So, let's check out what this particular drive has set on it by using `hdparm`:



[testbox]& hdparm /dev/hde

/dev/hde:
multcount = 16 (on)
IO_support = 0 (default 16-bit)
unmaskirq = 0 (off)
using_dma = 0 (off)
keepsettings = 0 (off)
readonly = 0 (off)
readahead = 256 (on)
geometry = 16383/255/63, sectors = 234493056, start = 0



...notice that DMA is turned off (in bold above). To enable DMA, simply type:



[testbox]& hdparm -d1 /dev/hde

/dev/hde:
setting using_dma to 1 (on)
using_dma = 1 (on)



Now that DMA is toggled on, let's see what we get for buffered disk and cache reads:



[testbox]& hdparm -Tt /dev/hde

/dev/hde:
Timing cached reads: 956 MB in 2.00 seconds = 476.95 MB/sec
Timing buffered disk reads: 170 MB in 3.03 seconds = 56.12 MB/sec



Not bad! ~56mb/sec is a much improvement over ~4mb/sec. Again, check the rest of your drives that you are using for your RAID setup and make sure DMA is enabled. Obviously DMA is just one of many options, but it's definitely the first and foremost thing you want to address right away.

We'll end there, even though we didn't get a whole lot covered. Next, I'll dive into setting up a software RAID-5 configuration using the `mdadm` tools, compare it with a RAID-0 configuration and decide which might be best for my needs, write a nice EXT3 filesystem onto it and benchmark away!