Friday 27 September 2013

Colour Grading and Sharpening

I loved working in a darkrooom, and is the thing I miss most about photography...nothing nowadays comes close, except maybe post production where you can tweak to your hearts content (or fix badly taken photos, as is too often the case!).

Recently, I've been tackling two basic post production topics: Colour Grading videos and Sharpening video and photos.

Colour Grading
Wow, what an interesting field.  The following quick video shows how much work it takes to make something simple look great, let alone the work that would be required for something like the hobbit!:

The "Camera original" shot (bit flat looking, but that is on purpose to enable better colour grading and maximise information capture i.e. dynamic range) was pretty darn good, until compared to the final.  What a difference!

I've been practising colour grading a little, but only in Photoshop and Lightroom as I don't have something like Premier.  Recently I've started looking at DaVinci Resolve.  Amazing for a free product, but is hellishly complex...can't wait to get my hands really dirty on it.

Different schools of thought, no definitive answer on whether its better to allow the camera to sharpen or sharpen in post.

Camera sharpening - the general theory is that sharpening in-camera is done before compression so that the sharpening is only done on the actual details rather than compression artefacts.

Post Sharpening - argument for this is that the sharpening algorithms used in camera is geared towards speed and not quality and you have more flexibility in post.  I don't know whether the first part about speed is true, but I certainly prefer the flexibility of post sharpening.

My own testing and prefered workflow has meant that I prefer to shoot with sharpening in-camera switched off and do any sharpening in post.  I like having full control over sharpening.  Same reason I shoot in Raw, I like having as much control over temp., black levels, white levels, etc etc.  I'm not a sports photographer, or shooting for clients who want to see results quickly, I just shoot because I like to.  I can afford to spend time in post and enjoy post production.

It's true that in most cases I use presets in post, which brings the sharpening to around what the camera would have done anyway, but it gives me scope to tweak the photos I really care about.  If it was set in camera I wouldn't get that option.

Tuesday 24 September 2013

Windows 8 Rant

So far the only good thing about Windows 8 is that I'm not using it on any of my desktops!

I thought it would be a good idea to upgrade my old laptop (Dell XPS M1330) before I handed it over to my daughter (just turned 7).  The laptop was running Vista Ultimate (:-0), so didn't think it was fair for her to deal with that.

So off I went; downloaded the Microsoft upgrade assistant and let it rip.  It came back to say, "yeah you're good to go but will have to reinstall a bunch of programs".  No problems for me as I wanted a fresh install anyway.  So went through the buying process; smooth.

The install process was just as easy, the only thing I think it asked me was whether I wanted to keep my settings to which I said no as I wanted a clean system.  The installer downloaded a 2 GB file, booted a couple of times and it was done (after I created my account that is).

The first frustration came when I tried to create an account for my daughter.  I wanted a Microsoft tied in account so that her settings would be saved in the "cloud" (what a dreadful term).  I also wanted protection and overview of her account.  So when creating her account, I told Microsoft her correct age.  When I logged in with her account on the laptop, Microsoft picked up on that and said, "you're a minor, please get parent to sign in"...all good.  So I signed in with my account and it said to verify that you are over 18 we'll carry out a $0.50 credit card transaction; no issues.  However when it redirected me to the payment site, the site was down.  After 30 mins, the site was still down.  This was becoming frustrating now as I couldn't get her email (google) working until I tied in her account.  it then started to work, so I got my CC out...another problem....I could not change country, so the address could only be american.  The frustration got to me at this point so I created a normal account and monitor on the local computer.

Yesterday I hit another frustration, and this is the one that has really got me wound up.  The Upgrade Assistant has installed a 32-bit OS not 64-bit!  Windows recognises the system as 64-bit capable, and I told it I didn't want any settings kept, but it still installed a 32-bit OS without giving me any indication it was doing that....what the hell!  If I had known it couldn't install 64-bit I would have stpped and done it differently.  To add to the frustration, it doesn't seem very easy to install the 64-bit version: .  Oh and this morning, my daughter pointed out that the system just shuts down completely every so often...I witnessed this myself...never happened on Vista previously.

I don't want to go into the usability aspects of windows 8 (an OS made for touch screens that is really, really non-intuitive) as that is another level of frustrations, as that would just make me cry.

Thursday 5 September 2013

Amahi 7 and Greyhole on HP Proliant (N54L) Microserver

I've had a long winded ride in getting Amahi 7 with Greyhole installed and working.

Some of the problems I faced:
  • Amahi 7 supports only installing from a DVD; I don't want a dvd drive
  • Amahi 7 USB install guide did not work for me
  • Amahi 7 does not have greyhole integrated yet, so a manual install has to be done
  • hda-greyhole did not work well with Fedora 19's Samba
  • After installing hda-greyhole and configuring samba to use it, I could no longer access the share and was getting the following errors:
Sep  5 01:48:23 amahi smbd[13392]:   Error loading module '/usr/lib64/samba/vfs/': /usr/lib64/samba/ version `SAMBA_4.0.0' not found (required by /usr/lib64/samba/vfs/
Sep  5 01:48:23 amahi smbd[13392]: [2013/09/05 01:48:23.379707,  0] ../source3/smbd/vfs.c:184(vfs_init_custom)
Sep  5 01:48:23 amahi smbd[13392]:   error probing vfs module 'greyhole': NT_STATUS_UNSUCCESSFUL
Sep  5 01:48:23 amahi smbd[13392]: [2013/09/05 01:48:23.379830,  0] ../source3/smbd/vfs.c:349(smbd_vfs_init)
Sep  5 01:48:23 amahi smbd[13392]:   smbd_vfs_init: vfs_init_custom failed for greyhole
Sep  5 01:48:23 amahi smbd[13392]: [2013/09/05 01:48:23.379946,  0] ../source3/smbd/service.c:632(make_connection_snum)
Sep  5 01:48:23 amahi smbd[13392]:   vfs_init failed for service Music
Sep  5 01:48:23 amahi smbd[13392]: [2013/09/05 01:48:23.389801,  0] ../lib/util/modules.c:56(load_module)

Here is a summarised list of how I got things working.  I've had to recreate all the steps here, leaving out all the incorrect attempts and dead ends and trying to patch them in an order that might work, so this might (in fact likely) to  have errors, so be careful, here be dragons.

Standard disclaimer applies; use the details here at your own risk, you could certainly destroy all your data and risk a bricked system! Also, this is outside of Amahi supported practices.

I run a headless system so all commands after the installation of Fedora, are run in a bash terminal.

I ran all commands as root (i.e. I executed sudo su – in the terminal first)

This assumes a fair amount of knowledge, like being able to use ping and edit files.

In a nutshell
  1. Upgrade Memory
  2. Update Bios
  3. Install Fedora 19
  4. Install Amahi
  5. Add Hard Drive
  6. Change Share locations
  7. Install Greyhole
Upgrade Memory
I installed more RAM (Corsair CMV8GX3M1A1333C9 Value Select 8GB (1x8GB) DDR3 1333 Mhz CL9 Mainstream Desktop Memory Module”) first after taking out the 2GB it came with as I prefer my RAM to be exactly the same in my servers.

The case is easy to take apart; after you take off all the cables connected to the motherboard (conveniently easy to get to as it is at the front) the motherboard comes out easily as it sits on a metal sliding base.  The only issue I had was taking off the two thumb screws holding the motherboard in place as they were too tight to remove by hand…but HP provide a handy L-shape star wrench to use.

Update Bios
The official bios for this board has been modded to support 6 ports of AHCI SATA 3gb/s.  You can find the modded bios here:

To create a bootable USB key with the bios just download the .rar file, extract, and execute RunMe.exe.

Install Fedora 19
Prepare your USB stick

Download Unetbootin:
Unetbootin is a standalone executable so doesn't need to be installed.  You will need admin rights.
Make sure you have a USB stick with enough space in your machine, then execute UNetbootin.
In the UNetbootin window for Distribution select Fedora 19_Live_x64

Make sure under type at the bottom, "USB Drive" is selected and that next to Drive it shows your USB stick, else you run the risk of overwriting your data.
Once UNetbootin finishes, don’t reboot, just exit.

Remove your USB stick and boot off it on your server.  By default it should be able to boot off USB, however you may need to enable booting off USB in the Bios if that is not already set .

Complete Fedora installation by following the instructions that appear.

Log on, turn on SSH and make available after reboot:
service sshd start
chkconfig sshd on

Make sure you can connect to the internet (ping for example).  If you can’t, try manually adding DNS servers to /etc/resolv.conf (you could add the openDNS servers for example: and

Install Amahi
To install Amahi execute:
rpm -Uvh
yum -y install mariadb-libs mariadb-server
yum -y install hda-ctl hda-platform

If all went well, your Amahi control panel ( should show your new install as running (might take a minute or two to catchcup)
Optionally you can turn off DHCP in your router at this stage

Add Hard Drive
I used parted for prepping my new 4TB hard drive, which appeared as /dev/sdb (read here:
parted /dev/sdb
at the parted prompt
mklabel gpt
mkpart primary ext4 4096s 100%
align-check opt 1

mkfs.ext4 -L greyhole01 /dev/sdb1
using the output from the above command, first backup /etc/fstab, and then add to /etc/fstab.
For me this meant I added the following to the bottom of my fstab:
UUID=e68bc54a-b795-4ce0-802f-213f4ab71e60 /var/hda/files/drives/drive1 ext4 defaults 1 2

Change Share Locations
I changed my default share paths from the drive that the OS lived on to my new drive using the Amahi web interface (after creating the following folder /var/hda/files/drives/drive1/shares/).  This is so I have plenty of space before greyhole can move files and as it is a drive in the pool, when Greyhole moves the files it should happen quicker if it decides to store a copy on the same drive (of course copies that need to be moved to other pooled drives will take more time).

Install Greyhole
As I’d already followed the installation instructions for installing greyhole on Amahi, before finding out that Amahi 7 didn’t work, I didn’t have to recreate the DB schema.  Here is what I had previously done:
yum -y install hda-greyhole
hda-create-db-and-user greyhole
mysql -u greyhole -pgreyhole  greyhole < /usr/share/greyhole/schema-mysql.sql

To upgrade greyhole, I removed hda-greyhole first:
yum remove hda-greyhole

and installed it from the greyhole repository
curl -o /etc/yum.repos.d/greyhole.repo
yum update greyhole

I edited the /etc/greyhole.conf file, adding in my storage_pool_drive:
storage_pool_drive = /var/hda/files/drives/drive1/gh, min_free: 10gb

...and changed the db_user and db_pass to (security minded people should change this at some point):
db_user = greyhole
db_pass = greyhole

In /etc/samba/smb.conf I added the following two lines to the shares I wanted to participate in the pool:
dfree command = /usr/bin/greyhole-dfree
vfs objects = greyhole

I then restarted Samba and greyhole
Service smb restart
Service greyhole restart

I now have a working Amahi 7 with greyhole system.  I’ve tested it and all seems to work very well.

In summary
The combination that did not work:
Samba: samba-4.0.9-1.fc19.x86_64
Amahi Greyhole: hda-greyhole-0.9.31-1.x86_64

The combination that does work:

I'm fairly sure I've made a few mistakes in creating this both in terms of steps taken to get Amahi+Greyhole running, and producing this document.  Please feel free to, constructively correct me :-)

Some links that gave me solutions and others that helped get me pointed in the right direction

Amahi Install related

Greyhole related

HD Installation related

General reading

Too many others to list

Tuesday 3 September 2013

HP Microserver N54L and Amahi

I bought a HP Microserver a little while ago to replace my 15 year oldish raid server.  I will be going from a full tower (around 60cm tall), 2.7 TB, 8 drive raid 5 array with 3ware raid controller (with OS on separate HD of course) running  Gentoo to a HP Microserver coming in at just 27 cms high, 4 TB on a single drive (for starters), with capacity for 3 other HDs (could have more if I use the 5.25 slot for more things change.  

Yesterday night I upgraded the paltry 2GB RAM it came with to 8GB RAM (system can handle 16 GB RAM apparently, might bump it up in future).  The server is super easy to take apart and get to the motherboard. The HD bay is also really convenient to get to, so when the 4TB HD comes through, it should be a breeze. 

I went for the Hitachi 4TB Deskstar (7K4000) in the end.  It was the top 4TB recommendation in Toms Hardware tests back in April 2012 and is at a price point within my budget now for the performance. This arrives today to be installed tonight after the bios firmware upgrade and Amahi install.

There are actually a few modded bios firmwares available for the HP (see below link).  The one I'm interested in is the one that allows the 6 USB ports to have AHCI enabled.  As it will be a server, I like to update the bios to the latest before I install any software, OS or otherwise, so this will be the first thing I do.  Luckily both the official and modded firmware are built to be done from a USB stick, so don't foresee any extra work there.

Once the firmware is updated I'm gonna be installing Amahi for all the NAS type things like sharing files, media, music and a whole host of other things.  

Amahi recently changed their install process from "install OS yourself, then install Amahi" to "here is an ISO that you need to burn to DVD and do the installation".  Currently that is the only supported method.  I don't have a DVD reader (and don't intend to have one as I have very little use for one) so can't use that method.   However, they have published a how-to for all us unsupported types, so hopefully will have the system up and running tonight.  Then I can copy across the existing 1.8 TB worth of data, backup to Bitcasa, and decomm. my 15 year old server...tears will be shed.