AccessControlException when pushing file to S3 bucket from GAE (on OSX)

This is one of those edge case scenarios that can drive people crazy.

I was trying to push a file from a Google App Engine app to an Amazon S3 bucket via the jetS3t API and it would keep coming back with an AccessControlException (access denied) kind of exception.

After quite a bit of digging around turns out the Mac default Java SDK will try to load a native library for the cryptographic needs of the S3 stuff and this is forbidden in GAE.

There's 2 workarounds apparently:

1. if you feel adventurous, (as I did) add this to your VM args from Properties > Run/Debug settings > Edit launch configuration options > Arguments: -D--enable_all_permissions=true
2. use another crypto library instead of the default (BouncyCastle is a common one, and it comes with the S3 API).  

Happy hacking!

EDIT (Dec 2nd 2011): Some news after more work on this, unfortunately solution 1) only fixes the problem locally - when you go and deploy you still get the same AccessControlException. Also, it appears that Google App Engine prevents you from specifying a custom crypto library so solution 2) is no good either. But bad news don't stop there, according to this thread: 

JetS3t is not compatibile with Google App Engine. Or the other way around. Because JetS3t uses a number of libraries and techniques that are not supported in the restricted execution environment of Google App Engine there is no easy way to remedy this.

Viable solutions seem to be:

1. This contraband version of the AWS SDK forked for app-engine
2. jclouds

I am in the process of trying some of this stuff. Will probably post something in a future post (if this stuff doesn't kill me first).

.NET Butchering --> Code Butchering

This is probably long overdue, I am changing the title of the blog to reflect the fact that I've been posting less and less .NET related stuff. This is somehow related to the fact that I've been trying to get more experience on non-microsoft technologies (mainly working on my pet open-source project), trying to practice my credo of language agnosticism.

How to ssh into Amazon EC2 Linux instance (and copy stuff over)

This is easy enough on an absolute scale but can be a bit of a nightmare if you're windows-oriented (as I unfortunately am ... but I am trying to snap out of it).

Here's a list of steps:

• Create amazon instance with a new key pair
• Save your private key to a known location (it's a .pem file you'll download in the process)
• Make sure the folder with the private key has appropriate access otherwise it won't work (e.g. on mac: chmod -R 700 path/to/pvtk).
• In the EC2 dashboard go the the security group settings for the instance and open ssh port 22
• Connect: ssh -i path/to/key/myKey.pem root@my-ec2-public-ip.amazonaws.com 

An alternative to referencing your key on the ssh command is to add it via ssh-add (on mac).

Another thing you might wanna do is to copy stuff over to the EC2 instance - you can use the scp command:

scp test/winning.txt root@my-ec2-public-ip.amazonaws.com:temp-files/

Good luck!

T-SQL Convert string to datetime

This is the kind of stuff you don't wanna waste time on - and yet ...

DECLARE @test varchar(10)
SET @test = '12/31/2011'

DECLARE @test_date datetime
SET @test_date = CONVERT(datetime, @test, 101)

Heads or Tails? or: how to lose money at the roulette table

I've always wondered how many heads/tails you'd get in a row if you could just keep flipping coins forever. The other day I was bored so I put together a stupid matlab script to try it out (leaving out the "forever" bit).

It's obvious that in theory you could get a never ending series of heads or tails, but in practice that is infinitely unlikely to happen, so I just wanted to get a feel for a "reasonable" figure in terms of how many repetitions one should expect to experience if one actually started tossing coins (or betting money on red/black at the roulette) for a very long time.

Out of 100 million series of coin tosses (where a series is defined by an uninterrupted streak of heads or tails) the longest streak of identical outcomes was 26 (I wanted to try with a billion series but was taking too long).

That means that if you go to a casino to make money doubling bets on red/black (the so-called Martingale strategy) and you are particularly unlucky you might have to flush out 20*(2^25) = 671,088,630 $ (a single bet is usually at least 20 $) on your last bet to actually make a profit (plus all the money you spent to get there).

If you play smart though, and you wait for 5 reds/blacks in a row before you start betting against that, then you can get away with a last bet of only 20*(2^20) = 20,971,520 $ (that's 20 millions!).

Disable DIV with jQuery

Here's a lame trick I use to disable divs:

// this works on IE

// disable
$('#myDiv').attr('disabled', 'disabled');


// enable
$('#myDiv').attr('disabled', '');

Disabling the whole div doesn't seem to disable inputs in chrome and firefox though, so here's the alternative version drilling down to each single input elements:

// this works everywhere but looks weird in IE


// disable
$('#myDiv :input').attr('disabled', 'disabled'); 


// enable
$('#myDiv :input').attr('disabled', '');

Good hacking.

Get list of OpenCL supported devices with jogamp.jocl

I recently got started on GPGPU, and the natural choice seemed to be OpenCL, since it's supported by both ATI and nVidia. The nice thing about OpenCL (other than being cross-platform) is also that it provides a layer of abstraction that allows you to use both CPUs and GPUs (not only GPUs as for nVidia's CUDA and AMD's Stream technologies).

I started investigating a few java wrappers (there's only a handful around) and ended up playing with jogamp.jocl.

This is just a snippet showing how to retrieve a list of OpenCL enabled devices on your machine:

// create context for all devices detected using default platform
CLContext context = CLContext.create();

// an array with available devices
CLDevice[] devices = context.getDevices();
         
for(int i=0; i < devices.length; i++)
{
   out.println("device-" + i + ": " + devices[i]);
} 

Goes without saying that if you don't see your GPU in the output it's time for some painful driver sweeping. I found that Snow Leopard works straightaway with both ATI and nVidia (MacOSX 10.6.x ships with OpenCl support), while windows can be a bit trickier to setup (as we all know, Catalyst software kinda sucks).

Just to give you a sneak-peek at what comes after, once you've had a look at the output then you can go ahead and select a device to create the queue(s) you'll use for sending data up to the devices:

// have a look at the output and select a device
CLDevice device = devices[0];

// create command queue on selected device.
CLCommandQueue queue = device.createCommandQueue();

You can see the entire code for the official jogamp.jocl  'Hello World' example here if you're curious.

Was Darwin Wrong?

In the beginning it was self-replicators.

Then it was self-replicators with metabolism (which kind of is the definition of life).

Now, to go from self-replicating blobs to human beings it takes a pretty good search algorithm.

Natural selection is that algorithm, and it happens to be the most robust search algorithm there is, beating hands down any other kind of human-designed search tool.

So, was Darwin wrong?

P.S. if you don't believe in facts then there is very little that can be done to convince you